2026-02-21T22:00:31.151 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T22:00:31.166 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T22:00:31.247 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078 branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle description: rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore/{alloc$/{stupid} base mem$/{normal-1} onode-segment$/{1M} write$/{v1/{compr$/{yes$/{zlib}} v1}}}} rados supported-random-distro$/{centos_latest}} email: null first_in_suite: false flavor: default job_id: '63078' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle ceph: conf: global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 30 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore onode segment size: 1024K bluestore write v2: false bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd objectstore: bluestore osd op num shards: 1 osd op num threads per shard: 2 osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 7c8cf961761b37084e637a305d42f77c52437802 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: distro-suffix: rockylinux-10 install: ceph: flavor: default sha1: 7c8cf961761b37084e637a305d42f77c52437802 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 53f8441172c46e34523029ab88f9db7a45514e84 owner: scheduled_nmordech@soko04.front.sepia.ceph.com priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 seed: 1998 sha1: 7c8cf961761b37084e637a305d42f77c52437802 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa suite_relpath: qa suite_repo: https://github.com/NitzanMordhai/ceph.git suite_sha1: 53f8441172c46e34523029ab88f9db7a45514e84 targets: trial060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5NQXVoY4h38zhNQ+yrKXz4nEI9dcd+t80hflziAP8JI6jej2Kq2lkI6rvza84qoGBqObQi3pvEIQAuyWr8rkM= tasks: - install: null - ceph: conf: osd: osd min pg log entries: 5 osd_fast_fail_on_connection_refused: false log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - osd_recovery.test_incomplete_pgs: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_21:55:34 tube: trial user: nmordech verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T22:00:31.247 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa; will attempt to use it 2026-02-21T22:00:31.248 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa/tasks 2026-02-21T22:00:31.248 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T22:00:31.249 INFO:teuthology.task.internal:Checking packages... 2026-02-21T22:00:31.249 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7c8cf961761b37084e637a305d42f77c52437802' 2026-02-21T22:00:31.249 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T22:00:31.249 INFO:teuthology.packaging:ref: None 2026-02-21T22:00:31.250 INFO:teuthology.packaging:tag: None 2026-02-21T22:00:31.250 INFO:teuthology.packaging:branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle 2026-02-21T22:00:31.250 INFO:teuthology.packaging:sha1: 7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:00:31.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle 2026-02-21T22:00:31.375 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5425.g7c8cf961 2026-02-21T22:00:31.377 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T22:00:31.388 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T22:00:31.388 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T22:00:31.402 INFO:teuthology.task.internal:Saving configuration 2026-02-21T22:00:31.411 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T22:00:31.423 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T22:00:31.501 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial060.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 21:56:55.502925', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5NQXVoY4h38zhNQ+yrKXz4nEI9dcd+t80hflziAP8JI6jej2Kq2lkI6rvza84qoGBqObQi3pvEIQAuyWr8rkM='} 2026-02-21T22:00:31.501 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T22:00:31.513 INFO:teuthology.task.internal:roles: ubuntu@trial060.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2026-02-21T22:00:31.513 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T22:00:31.600 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x759198ff7600>, signals=[15]) 2026-02-21T22:00:31.600 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T22:00:31.612 INFO:teuthology.task.internal:Opening connections... 2026-02-21T22:00:31.612 DEBUG:teuthology.task.internal:connecting to ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:00:31.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T22:00:31.676 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T22:00:31.687 DEBUG:teuthology.orchestra.run.trial060:> uname -m 2026-02-21T22:00:31.748 INFO:teuthology.orchestra.run.trial060.stdout:x86_64 2026-02-21T22:00:31.748 DEBUG:teuthology.orchestra.run.trial060:> cat /etc/os-release 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:NAME="CentOS Stream" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:VERSION="9" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:ID="centos" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:ID_LIKE="rhel fedora" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:VERSION_ID="9" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:PLATFORM_ID="platform:el9" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:ANSI_COLOR="0;31" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:LOGO="fedora-logo-icon" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:HOME_URL="https://centos.org/" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-21T22:00:31.804 INFO:teuthology.orchestra.run.trial060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-21T22:00:31.805 INFO:teuthology.lock.ops:Updating trial060.front.sepia.ceph.com on lock server 2026-02-21T22:00:31.885 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T22:00:31.903 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T22:00:31.914 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T22:00:31.915 DEBUG:teuthology.orchestra.run.trial060:> test '!' -e /home/ubuntu/cephtest 2026-02-21T22:00:31.929 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T22:00:31.940 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T22:00:31.941 DEBUG:teuthology.orchestra.run.trial060:> test -z $(ls -A /var/lib/ceph) 2026-02-21T22:00:31.986 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T22:00:32.088 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T22:00:32.109 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-21T22:00:32.109 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-21T22:00:32.109 DEBUG:teuthology.orchestra.run.trial060:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T22:00:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:00:32.124 DEBUG:teuthology.orchestra.run.trial060:> uname -r 2026-02-21T22:00:32.180 INFO:teuthology.orchestra.run.trial060.stdout:5.14.0-681.el9.x86_64 2026-02-21T22:00:32.181 INFO:teuthology.task.kernel:Running kernel on trial060: 5.14.0-681.el9.x86_64 2026-02-21T22:00:32.181 DEBUG:teuthology.orchestra.run.trial060:> sudo yum install -y kernel 2026-02-21T22:00:33.407 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-21T22:00:39.514 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 27 MB 00:05 2026-02-21T22:00:43.127 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 7.8 MB 00:01 2026-02-21T22:00:43.949 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - Extras packages 184 kB/s | 20 kB 00:00 2026-02-21T22:00:44.660 INFO:teuthology.orchestra.run.trial060.stdout:Extra Packages for Enterprise Linux 31 MB/s | 20 MB 00:00 2026-02-21T22:00:47.871 INFO:teuthology.orchestra.run.trial060.stdout:lab-extras 1.5 MB/s | 50 kB 00:00 2026-02-21T22:00:48.880 INFO:teuthology.orchestra.run.trial060.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-21T22:00:48.880 INFO:teuthology.orchestra.run.trial060.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-21T22:00:48.899 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:00:48.900 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:00:48.900 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:00:48.923 DEBUG:teuthology.orchestra.run.trial060:> echo no | sudo yum reinstall kernel || true 2026-02-21T22:00:49.220 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 10:00:47 PM UTC. 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout:Reinstalling: 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:00:49.300 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:00:49.301 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 803 k 2026-02-21T22:00:49.301 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 0 2026-02-21T22:00:49.301 INFO:teuthology.orchestra.run.trial060.stderr:Operation aborted. 2026-02-21T22:00:49.320 INFO:teuthology.orchestra.run.trial060.stdout:Is this ok [y/N]: 2026-02-21T22:00:49.320 DEBUG:teuthology.orchestra.run.trial060:> sudo yum reinstall -y kernel || true 2026-02-21T22:00:49.619 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 10:00:47 PM UTC. 2026-02-21T22:00:49.699 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:00:49.699 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:Reinstalling: 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 803 k 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 0 2026-02-21T22:00:49.700 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:00:50.247 INFO:teuthology.orchestra.run.trial060.stdout:kernel-5.14.0-681.el9.x86_64.rpm 3.7 MB/s | 803 kB 00:00 2026-02-21T22:00:50.247 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:00:50.247 INFO:teuthology.orchestra.run.trial060.stdout:Total 1.4 MB/s | 803 kB 00:00 2026-02-21T22:00:50.257 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:00:50.290 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:00:50.290 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:00:50.294 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:00:50.294 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:00:50.318 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:00:50.322 INFO:teuthology.orchestra.run.trial060.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T22:00:50.322 INFO:teuthology.orchestra.run.trial060.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T22:00:50.436 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T22:00:50.436 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T22:00:50.488 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T22:00:50.488 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:00:50.489 INFO:teuthology.orchestra.run.trial060.stdout:Reinstalled: 2026-02-21T22:00:50.489 INFO:teuthology.orchestra.run.trial060.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-21T22:00:50.489 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:00:50.489 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:00:50.515 DEBUG:teuthology.orchestra.run.trial060:> rpm -q kernel | sort -rV | head -n 1 2026-02-21T22:00:50.578 INFO:teuthology.orchestra.run.trial060.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-21T22:00:50.579 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-21T22:00:50.580 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T22:00:50.580 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T22:00:50.580 DEBUG:teuthology.orchestra.run.trial060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T22:00:50.643 INFO:teuthology.orchestra.run.trial060.stdout:ttyS1 2026-02-21T22:00:50.663 DEBUG:teuthology.parallel:result is None 2026-02-21T22:00:50.663 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T22:00:50.681 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T22:00:50.682 DEBUG:teuthology.orchestra.run.trial060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T22:00:50.719 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T22:00:50.730 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T22:00:50.744 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T22:00:50.744 DEBUG:teuthology.orchestra.run.trial060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T22:00:50.784 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T22:00:50.795 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T22:00:50.796 DEBUG:teuthology.orchestra.run.trial060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T22:00:50.839 INFO:teuthology.orchestra.run.trial060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T22:00:50.848 INFO:teuthology.orchestra.run.trial060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T22:00:50.850 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T22:00:50.864 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T22:00:50.864 DEBUG:teuthology.orchestra.run.trial060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T22:00:50.915 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T22:00:50.930 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T22:00:50.930 DEBUG:teuthology.orchestra.run.trial060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T22:00:50.971 DEBUG:teuthology.orchestra.run.trial060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:00:51.111 DEBUG:teuthology.orchestra.run.trial060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:00:51.134 DEBUG:teuthology.orchestra.run.trial060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:00:51.265 DEBUG:teuthology.orchestra.run.trial060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:00:51.288 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:00:51.288 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T22:00:51.351 DEBUG:teuthology.orchestra.run.trial060:> sudo service rsyslog restart 2026-02-21T22:00:51.419 INFO:teuthology.orchestra.run.trial060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T22:00:51.724 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T22:00:51.738 INFO:teuthology.task.internal:Starting timer... 2026-02-21T22:00:51.738 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T22:00:51.762 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T22:00:51.782 DEBUG:teuthology.orchestra.run.trial060:> sudo service auditd rotate 2026-02-21T22:00:51.811 INFO:teuthology.orchestra.run.trial060.stdout:Rotating logs: 2026-02-21T22:00:51.812 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T22:00:51.813 DEBUG:teuthology.orchestra.run.trial060:> /usr/sbin/getenforce 2026-02-21T22:00:51.870 INFO:teuthology.orchestra.run.trial060.stdout:Permissive 2026-02-21T22:00:51.870 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial060.front.sepia.ceph.com': 'permissive'} 2026-02-21T22:00:51.871 DEBUG:teuthology.orchestra.run.trial060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T22:00:51.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:00:51.934 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T22:00:51.934 DEBUG:teuthology.orchestra.run.trial060:> sudo /usr/sbin/setenforce permissive 2026-02-21T22:00:51.999 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T22:00:52.015 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T22:00:52.022 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T22:00:52.022 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial060.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T22:03:17.306 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial060.front.sepia.ceph.com')] 2026-02-21T22:03:17.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-21T22:03:17.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T22:03:17.392 DEBUG:teuthology.orchestra.run.trial060:> true 2026-02-21T22:03:17.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-21T22:03:17.466 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T22:03:17.480 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T22:03:17.480 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T22:03:17.480 DEBUG:teuthology.orchestra.run.trial060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T22:03:17.536 INFO:teuthology.orchestra.run.trial060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-21T22:03:17.548 INFO:teuthology.orchestra.run.trial060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-21T22:03:17.570 INFO:teuthology.orchestra.run.trial060.stderr:sudo: ntpd: command not found 2026-02-21T22:03:17.581 INFO:teuthology.orchestra.run.trial060.stdout:506 Cannot talk to daemon 2026-02-21T22:03:17.593 INFO:teuthology.orchestra.run.trial060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-21T22:03:17.605 INFO:teuthology.orchestra.run.trial060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-21T22:03:17.648 INFO:teuthology.orchestra.run.trial060.stderr:bash: line 1: ntpq: command not found 2026-02-21T22:03:17.650 INFO:teuthology.orchestra.run.trial060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T22:03:17.650 INFO:teuthology.orchestra.run.trial060.stdout:=============================================================================== 2026-02-21T22:03:17.650 INFO:teuthology.run_tasks:Running task install... 2026-02-21T22:03:17.662 DEBUG:teuthology.task.install:project ceph 2026-02-21T22:03:17.662 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}} 2026-02-21T22:03:17.662 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'} 2026-02-21T22:03:17.662 INFO:teuthology.task.install:Using flavor: default 2026-02-21T22:03:17.666 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T22:03:17.667 INFO:teuthology.task.install:extra packages: [] 2026-02-21T22:03:17.667 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'tag': None, 'wait_for_package': False} 2026-02-21T22:03:17.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:03:17.816 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/ 2026-02-21T22:03:17.816 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5425.g7c8cf961 2026-02-21T22:03:18.013 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T22:03:18.013 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:03:18.013 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T22:03:18.039 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-21T22:03:18.039 DEBUG:teuthology.orchestra.run.trial060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7c8cf961761b37084e637a305d42f77c52437802/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T22:03:18.104 DEBUG:teuthology.orchestra.run.trial060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T22:03:18.181 DEBUG:teuthology.orchestra.run.trial060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T22:03:18.245 INFO:teuthology.orchestra.run.trial060.stdout:check_obsoletes = 1 2026-02-21T22:03:18.246 DEBUG:teuthology.orchestra.run.trial060:> sudo yum clean all 2026-02-21T22:03:18.407 INFO:teuthology.orchestra.run.trial060.stdout:41 files removed 2026-02-21T22:03:18.422 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-radosgw 2026-02-21T22:03:18.940 INFO:teuthology.orchestra.run.trial060.stdout:ceph packages for x86_64 250 kB/s | 93 kB 00:00 2026-02-21T22:03:19.281 INFO:teuthology.orchestra.run.trial060.stdout:ceph noarch packages 78 kB/s | 25 kB 00:00 2026-02-21T22:03:19.705 INFO:teuthology.orchestra.run.trial060.stdout:ceph source packages 5.1 kB/s | 2.1 kB 00:00 2026-02-21T22:03:20.446 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-21T22:03:23.608 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-21T22:03:26.472 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - CRB 20 MB/s | 7.8 MB 00:00 2026-02-21T22:03:28.209 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-21T22:03:28.748 INFO:teuthology.orchestra.run.trial060.stdout:Extra Packages for Enterprise Linux 43 MB/s | 20 MB 00:00 2026-02-21T22:03:32.026 INFO:teuthology.orchestra.run.trial060.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-21T22:03:33.223 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repository Size 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout: ceph-radosgw x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 13 M 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout:Upgrading: 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout: librados2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 3.5 M 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout: librbd1 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 2.8 M 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-21T22:03:33.224 INFO:teuthology.orchestra.run.trial060.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: ceph-base x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 5.2 M 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: ceph-common x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 24 M 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: ceph-selinux x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 25 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 908 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libradosstriper1 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 248 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: librgw2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 6.5 M 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-argparse x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 45 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-common x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 185 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: python3-cephfs x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 203 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: python3-rados x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 324 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: python3-rbd x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 306 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: python3-rgw x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 100 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-21T22:03:33.225 INFO:teuthology.orchestra.run.trial060.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:Installing weak dependencies: 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-daemon x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 37 k 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 26 k 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:Install 35 Packages 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:Upgrade 2 Packages 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 68 M 2026-02-21T22:03:33.226 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:03:33.977 INFO:teuthology.orchestra.run.trial060.stdout:(1/37): ceph-base-20.3.0-5425.g7c8cf961.el9.x86 16 MB/s | 5.2 MB 00:00 2026-02-21T22:03:34.014 INFO:teuthology.orchestra.run.trial060.stdout:(2/37): ceph-selinux-20.3.0-5425.g7c8cf961.el9. 675 kB/s | 25 kB 00:00 2026-02-21T22:03:34.052 INFO:teuthology.orchestra.run.trial060.stdout:(3/37): libcephfs-daemon-20.3.0-5425.g7c8cf961. 992 kB/s | 37 kB 00:00 2026-02-21T22:03:34.090 INFO:teuthology.orchestra.run.trial060.stdout:(4/37): libcephfs-proxy2-20.3.0-5425.g7c8cf961. 665 kB/s | 26 kB 00:00 2026-02-21T22:03:34.148 INFO:teuthology.orchestra.run.trial060.stdout:(5/37): libcephfs2-20.3.0-5425.g7c8cf961.el9.x8 16 MB/s | 908 kB 00:00 2026-02-21T22:03:34.194 INFO:teuthology.orchestra.run.trial060.stdout:(6/37): libradosstriper1-20.3.0-5425.g7c8cf961. 5.2 MB/s | 248 kB 00:00 2026-02-21T22:03:34.590 INFO:teuthology.orchestra.run.trial060.stdout:(7/37): librgw2-20.3.0-5425.g7c8cf961.el9.x86_6 16 MB/s | 6.5 MB 00:00 2026-02-21T22:03:34.628 INFO:teuthology.orchestra.run.trial060.stdout:(8/37): python3-ceph-argparse-20.3.0-5425.g7c8c 1.2 MB/s | 45 kB 00:00 2026-02-21T22:03:34.688 INFO:teuthology.orchestra.run.trial060.stdout:(9/37): ceph-common-20.3.0-5425.g7c8cf961.el9.x 23 MB/s | 24 MB 00:01 2026-02-21T22:03:34.689 INFO:teuthology.orchestra.run.trial060.stdout:(10/37): python3-ceph-common-20.3.0-5425.g7c8cf 3.0 MB/s | 185 kB 00:00 2026-02-21T22:03:34.700 INFO:teuthology.orchestra.run.trial060.stdout:(11/37): python3-cephfs-20.3.0-5425.g7c8cf961.e 16 MB/s | 203 kB 00:00 2026-02-21T22:03:34.709 INFO:teuthology.orchestra.run.trial060.stdout:(12/37): python3-rados-20.3.0-5425.g7c8cf961.el 16 MB/s | 324 kB 00:00 2026-02-21T22:03:34.727 INFO:teuthology.orchestra.run.trial060.stdout:(13/37): python3-rbd-20.3.0-5425.g7c8cf961.el9. 11 MB/s | 306 kB 00:00 2026-02-21T22:03:34.731 INFO:teuthology.orchestra.run.trial060.stdout:(14/37): python3-rgw-20.3.0-5425.g7c8cf961.el9. 4.4 MB/s | 100 kB 00:00 2026-02-21T22:03:34.757 INFO:teuthology.orchestra.run.trial060.stdout:(15/37): ceph-radosgw-20.3.0-5425.g7c8cf961.el9 12 MB/s | 13 MB 00:01 2026-02-21T22:03:35.341 INFO:teuthology.orchestra.run.trial060.stdout:(16/37): mailcap-2.1.49-5.el9.noarch.rpm 54 kB/s | 33 kB 00:00 2026-02-21T22:03:35.511 INFO:teuthology.orchestra.run.trial060.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 611 kB/s | 104 kB 00:00 2026-02-21T22:03:35.574 INFO:teuthology.orchestra.run.trial060.stdout:(18/37): pciutils-3.7.0-7.el9.x86_64.rpm 114 kB/s | 93 kB 00:00 2026-02-21T22:03:35.752 INFO:teuthology.orchestra.run.trial060.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 681 kB/s | 164 kB 00:00 2026-02-21T22:03:35.835 INFO:teuthology.orchestra.run.trial060.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 544 kB/s | 45 kB 00:00 2026-02-21T22:03:36.168 INFO:teuthology.orchestra.run.trial060.stdout:(21/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 269 kB/s | 160 kB 00:00 2026-02-21T22:03:36.628 INFO:teuthology.orchestra.run.trial060.stdout:(22/37): librdkafka-1.6.1-102.el9.x86_64.rpm 836 kB/s | 662 kB 00:00 2026-02-21T22:03:36.746 INFO:teuthology.orchestra.run.trial060.stdout:(23/37): nvme-cli-2.16-1.el9.x86_64.rpm 585 kB/s | 1.2 MB 00:02 2026-02-21T22:03:36.829 INFO:teuthology.orchestra.run.trial060.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 445 kB/s | 37 kB 00:00 2026-02-21T22:03:36.884 INFO:teuthology.orchestra.run.trial060.stdout:(25/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 408 kB/s | 292 kB 00:00 2026-02-21T22:03:36.912 INFO:teuthology.orchestra.run.trial060.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 801 kB/s | 66 kB 00:00 2026-02-21T22:03:36.948 INFO:teuthology.orchestra.run.trial060.stdout:(27/37): qatlib-25.08.0-2.el9.x86_64.rpm 749 kB/s | 240 kB 00:00 2026-02-21T22:03:37.111 INFO:teuthology.orchestra.run.trial060.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 152 kB/s | 25 kB 00:00 2026-02-21T22:03:37.221 INFO:teuthology.orchestra.run.trial060.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2026-02-21T22:03:37.237 INFO:teuthology.orchestra.run.trial060.stdout:(30/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 871 kB/s | 308 kB 00:00 2026-02-21T22:03:37.299 INFO:teuthology.orchestra.run.trial060.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 855 kB/s | 67 kB 00:00 2026-02-21T22:03:37.393 INFO:teuthology.orchestra.run.trial060.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2026-02-21T22:03:37.426 INFO:teuthology.orchestra.run.trial060.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2026-02-21T22:03:37.455 INFO:teuthology.orchestra.run.trial060.stdout:(34/37): libarrow-9.0.0-15.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2026-02-21T22:03:37.518 INFO:teuthology.orchestra.run.trial060.stdout:(35/37): thrift-0.15.0-4.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2026-02-21T22:03:38.039 INFO:teuthology.orchestra.run.trial060.stdout:(36/37): librados2-20.3.0-5425.g7c8cf961.el9.x8 5.7 MB/s | 3.5 MB 00:00 2026-02-21T22:03:38.070 INFO:teuthology.orchestra.run.trial060.stdout:(37/37): librbd1-20.3.0-5425.g7c8cf961.el9.x86_ 4.6 MB/s | 2.8 MB 00:00 2026-02-21T22:03:38.073 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:03:38.073 INFO:teuthology.orchestra.run.trial060.stdout:Total 14 MB/s | 68 MB 00:04 2026-02-21T22:03:38.168 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:03:38.188 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:03:38.188 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:03:38.460 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:03:38.460 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:03:38.771 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:03:38.932 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-21T22:03:38.936 INFO:teuthology.orchestra.run.trial060.stdout: Upgrading : librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/39 2026-02-21T22:03:38.989 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/39 2026-02-21T22:03:38.990 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/39 2026-02-21T22:03:39.003 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/39 2026-02-21T22:03:39.005 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephfs-daemon-2:20.3.0-5425.g7c8cf961.el9.x86_6 4/39 2026-02-21T22:03:39.006 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_6 5/39 2026-02-21T22:03:39.034 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_6 5/39 2026-02-21T22:03:39.037 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 6/39 2026-02-21T22:03:39.046 INFO:teuthology.orchestra.run.trial060.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-21T22:03:39.049 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-21T22:03:39.052 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-21T22:03:39.056 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-21T22:03:39.067 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-ceph-argparse-2:20.3.0-5425.g7c8cf961.el9. 11/39 2026-02-21T22:03:39.079 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 12/39 2026-02-21T22:03:39.080 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_6 13/39 2026-02-21T22:03:39.103 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_6 13/39 2026-02-21T22:03:39.108 INFO:teuthology.orchestra.run.trial060.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-21T22:03:39.139 INFO:teuthology.orchestra.run.trial060.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-21T22:03:39.144 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-21T22:03:39.151 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-21T22:03:39.154 INFO:teuthology.orchestra.run.trial060.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-21T22:03:39.158 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-21T22:03:39.287 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-21T22:03:39.288 INFO:teuthology.orchestra.run.trial060.stdout: Upgrading : librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 21/39 2026-02-21T22:03:39.315 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 21/39 2026-02-21T22:03:39.317 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 22/39 2026-02-21T22:03:39.323 INFO:teuthology.orchestra.run.trial060.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-21T22:03:39.623 INFO:teuthology.orchestra.run.trial060.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-21T22:03:39.624 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 25/39 2026-02-21T22:03:39.640 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 25/39 2026-02-21T22:03:39.646 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 26/39 2026-02-21T22:03:39.651 INFO:teuthology.orchestra.run.trial060.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-21T22:03:39.653 INFO:teuthology.orchestra.run.trial060.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-21T22:03:39.654 INFO:teuthology.orchestra.run.trial060.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-21T22:03:39.672 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-21T22:03:39.788 INFO:teuthology.orchestra.run.trial060.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-21T22:03:39.791 INFO:teuthology.orchestra.run.trial060.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-21T22:03:40.081 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-21T22:03:40.081 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T22:03:40.081 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:40.125 INFO:teuthology.orchestra.run.trial060.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-21T22:03:40.127 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-ceph-common-2:20.3.0-5425.g7c8cf961.el9.x8 33/39 2026-02-21T22:03:41.280 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 34/39 2026-02-21T22:03:41.282 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 34/39 2026-02-21T22:03:41.552 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 34/39 2026-02-21T22:03:41.556 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 35/39 2026-02-21T22:03:41.604 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 35/39 2026-02-21T22:03:41.604 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-21T22:03:41.604 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-21T22:03:41.604 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:41.607 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 36/39 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 36/39 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /sys 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /proc 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /mnt 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /var/tmp 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /home 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /root 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /tmp 2026-02-21T22:03:46.136 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:46.748 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 37/39 2026-02-21T22:03:46.772 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 37/39 2026-02-21T22:03:46.772 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:03:46.772 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-21T22:03:46.772 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T22:03:46.772 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T22:03:46.772 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:46.774 INFO:teuthology.orchestra.run.trial060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-21T22:03:46.786 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-21T22:03:46.786 INFO:teuthology.orchestra.run.trial060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T22:03:46.842 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T22:03:46.842 INFO:teuthology.orchestra.run.trial060.stdout:Creating group 'qat' with GID 992. 2026-02-21T22:03:46.842 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 4/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephfs-daemon-2:20.3.0-5425.g7c8cf961.el9.x86_6 5/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_6 6/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 7/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_6 8/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 9/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5425.g7c8cf961.el9. 10/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-ceph-common-2:20.3.0-5425.g7c8cf961.el9.x8 11/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 12/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 13/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 14/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 15/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-21T22:03:47.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 36/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-21T22:03:47.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 38/39 2026-02-21T22:03:47.208 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T22:03:47.208 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:47.208 INFO:teuthology.orchestra.run.trial060.stdout:Upgraded: 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-daemon-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-argparse-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: re2-1:20211101-20.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:47.209 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:03:47.251 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-test 2026-02-21T22:03:47.565 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:15 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:03:47.648 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:03:47.648 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:03:47.648 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repository Size 2026-02-21T22:03:47.648 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:03:47.648 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: ceph-test x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 78 M 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: libcephsqlite x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 159 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:Install 10 Packages 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 82 M 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 339 M 2026-02-21T22:03:47.649 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:03:48.468 INFO:teuthology.orchestra.run.trial060.stdout:(1/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 7.8 MB/s | 2.1 MB 00:00 2026-02-21T22:03:48.512 INFO:teuthology.orchestra.run.trial060.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 525 kB/s | 23 kB 00:00 2026-02-21T22:03:48.549 INFO:teuthology.orchestra.run.trial060.stdout:(3/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.5 MB/s | 94 kB 00:00 2026-02-21T22:03:48.558 INFO:teuthology.orchestra.run.trial060.stdout:(4/10): libcephsqlite-20.3.0-5425.g7c8cf961.el9 441 kB/s | 159 kB 00:00 2026-02-21T22:03:48.569 INFO:teuthology.orchestra.run.trial060.stdout:(5/10): libxslt-1.1.34-12.el9.x86_64.rpm 11 MB/s | 233 kB 00:00 2026-02-21T22:03:48.582 INFO:teuthology.orchestra.run.trial060.stdout:(6/10): socat-1.7.4.1-8.el9.x86_64.rpm 24 MB/s | 303 kB 00:00 2026-02-21T22:03:48.585 INFO:teuthology.orchestra.run.trial060.stdout:(7/10): protobuf-3.14.0-17.el9.x86_64.rpm 36 MB/s | 1.0 MB 00:00 2026-02-21T22:03:48.586 INFO:teuthology.orchestra.run.trial060.stdout:(8/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 13 MB/s | 64 kB 00:00 2026-02-21T22:03:48.718 INFO:teuthology.orchestra.run.trial060.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 921 kB/s | 122 kB 00:00 2026-02-21T22:03:58.925 INFO:teuthology.orchestra.run.trial060.stdout:(10/10): ceph-test-20.3.0-5425.g7c8cf961.el9.x8 7.3 MB/s | 78 MB 00:10 2026-02-21T22:03:58.927 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:03:58.927 INFO:teuthology.orchestra.run.trial060.stdout:Total 7.3 MB/s | 82 MB 00:11 2026-02-21T22:03:58.990 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:03:59.002 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:03:59.002 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:03:59.287 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:03:59.287 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:03:59.598 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:03:59.607 INFO:teuthology.orchestra.run.trial060.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-21T22:03:59.615 INFO:teuthology.orchestra.run.trial060.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-21T22:03:59.620 INFO:teuthology.orchestra.run.trial060.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-21T22:03:59.624 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-21T22:03:59.630 INFO:teuthology.orchestra.run.trial060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-21T22:03:59.632 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-21T22:03:59.642 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-21T22:03:59.650 INFO:teuthology.orchestra.run.trial060.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-21T22:03:59.650 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 9/10 2026-02-21T22:04:03.091 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 9/10 2026-02-21T22:04:03.109 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 10/10 2026-02-21T22:04:03.459 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 10/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-21T22:04:03.460 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-21T22:04:03.514 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-21T22:04:03.514 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:03.514 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:03.514 INFO:teuthology.orchestra.run.trial060.stdout: ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:03.514 INFO:teuthology.orchestra.run.trial060.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:03.515 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:03.553 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph 2026-02-21T22:04:03.863 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:31 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:03.949 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:03.951 INFO:teuthology.orchestra.run.trial060.stdout:======================================================================================= 2026-02-21T22:04:03.951 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repository Size 2026-02-21T22:04:03.951 INFO:teuthology.orchestra.run.trial060.stdout:======================================================================================= 2026-02-21T22:04:03.951 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 6.5 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 57 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mds x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 2.4 M 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 940 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 297 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mon x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 5.0 M 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 167 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd-classic x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 16 M 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd-crimson x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 19 M 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 17 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: cephadm noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 1.0 M 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-21T22:04:03.952 INFO:teuthology.orchestra.run.trial060.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-21T22:04:03.953 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-21T22:04:03.954 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout:Installing weak dependencies: 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 184 k 2026-02-21T22:04:03.955 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 51 M 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 7.4 M 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 22 k 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-rook noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 50 k 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: ceph-volume noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 301 k 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout:======================================================================================= 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout:Install 96 Packages 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 158 M 2026-02-21T22:04:03.956 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 732 M 2026-02-21T22:04:03.957 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:04.823 INFO:teuthology.orchestra.run.trial060.stdout:(1/96): ceph-20.3.0-5425.g7c8cf961.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2026-02-21T22:04:05.315 INFO:teuthology.orchestra.run.trial060.stdout:(2/96): ceph-mds-20.3.0-5425.g7c8cf961.el9.x86_ 3.1 MB/s | 2.4 MB 00:00 2026-02-21T22:04:05.376 INFO:teuthology.orchestra.run.trial060.stdout:(3/96): ceph-osd-20.3.0-5425.g7c8cf961.el9.x86_ 2.6 MB/s | 167 kB 00:00 2026-02-21T22:04:05.676 INFO:teuthology.orchestra.run.trial060.stdout:(4/96): ceph-mon-20.3.0-5425.g7c8cf961.el9.x86_ 5.8 MB/s | 5.0 MB 00:00 2026-02-21T22:04:07.074 INFO:teuthology.orchestra.run.trial060.stdout:(5/96): ceph-osd-classic-20.3.0-5425.g7c8cf961. 9.2 MB/s | 16 MB 00:01 2026-02-21T22:04:07.075 INFO:teuthology.orchestra.run.trial060.stdout:(6/96): ceph-mgr-20.3.0-5425.g7c8cf961.el9.x86_ 373 kB/s | 940 kB 00:02 2026-02-21T22:04:07.120 INFO:teuthology.orchestra.run.trial060.stdout:(7/96): ceph-grafana-dashboards-20.3.0-5425.g7c 1.2 MB/s | 57 kB 00:00 2026-02-21T22:04:07.341 INFO:teuthology.orchestra.run.trial060.stdout:(8/96): ceph-mgr-cephadm-20.3.0-5425.g7c8cf961. 694 kB/s | 184 kB 00:00 2026-02-21T22:04:07.878 INFO:teuthology.orchestra.run.trial060.stdout:(9/96): ceph-osd-crimson-20.3.0-5425.g7c8cf961. 8.7 MB/s | 19 MB 00:02 2026-02-21T22:04:07.931 INFO:teuthology.orchestra.run.trial060.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5425.g7c8cf9 423 kB/s | 22 kB 00:00 2026-02-21T22:04:07.990 INFO:teuthology.orchestra.run.trial060.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5425.g7c8 4.9 MB/s | 297 kB 00:00 2026-02-21T22:04:08.045 INFO:teuthology.orchestra.run.trial060.stdout:(12/96): ceph-mgr-rook-20.3.0-5425.g7c8cf961.el 922 kB/s | 50 kB 00:00 2026-02-21T22:04:08.100 INFO:teuthology.orchestra.run.trial060.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5425.g7c 319 kB/s | 17 kB 00:00 2026-02-21T22:04:08.160 INFO:teuthology.orchestra.run.trial060.stdout:(14/96): ceph-volume-20.3.0-5425.g7c8cf961.el9. 4.9 MB/s | 301 kB 00:00 2026-02-21T22:04:08.334 INFO:teuthology.orchestra.run.trial060.stdout:(15/96): cephadm-20.3.0-5425.g7c8cf961.el9.noar 5.9 MB/s | 1.0 MB 00:00 2026-02-21T22:04:08.471 INFO:teuthology.orchestra.run.trial060.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.5 MB/s | 351 kB 00:00 2026-02-21T22:04:08.492 INFO:teuthology.orchestra.run.trial060.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.9 MB/s | 40 kB 00:00 2026-02-21T22:04:08.512 INFO:teuthology.orchestra.run.trial060.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 3.4 MB/s | 72 kB 00:00 2026-02-21T22:04:08.554 INFO:teuthology.orchestra.run.trial060.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 19 MB/s | 794 kB 00:00 2026-02-21T22:04:08.577 INFO:teuthology.orchestra.run.trial060.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 7.7 MB/s | 184 kB 00:00 2026-02-21T22:04:08.601 INFO:teuthology.orchestra.run.trial060.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 11 MB/s | 253 kB 00:00 2026-02-21T22:04:08.659 INFO:teuthology.orchestra.run.trial060.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 21 MB/s | 1.2 MB 00:00 2026-02-21T22:04:08.682 INFO:teuthology.orchestra.run.trial060.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.7 MB/s | 106 kB 00:00 2026-02-21T22:04:08.704 INFO:teuthology.orchestra.run.trial060.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 5.9 MB/s | 135 kB 00:00 2026-02-21T22:04:08.727 INFO:teuthology.orchestra.run.trial060.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 6.5 MB/s | 150 kB 00:00 2026-02-21T22:04:08.750 INFO:teuthology.orchestra.run.trial060.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 5.4 MB/s | 126 kB 00:00 2026-02-21T22:04:08.772 INFO:teuthology.orchestra.run.trial060.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 9.3 MB/s | 218 kB 00:00 2026-02-21T22:04:08.796 INFO:teuthology.orchestra.run.trial060.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 7.6 MB/s | 182 kB 00:00 2026-02-21T22:04:08.818 INFO:teuthology.orchestra.run.trial060.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 12 MB/s | 266 kB 00:00 2026-02-21T22:04:09.082 INFO:teuthology.orchestra.run.trial060.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 113 kB/s | 30 kB 00:00 2026-02-21T22:04:09.680 INFO:teuthology.orchestra.run.trial060.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 5.0 MB/s | 3.0 MB 00:00 2026-02-21T22:04:09.759 INFO:teuthology.orchestra.run.trial060.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 188 kB/s | 15 kB 00:00 2026-02-21T22:04:09.841 INFO:teuthology.orchestra.run.trial060.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.9 MB/s | 246 kB 00:00 2026-02-21T22:04:09.923 INFO:teuthology.orchestra.run.trial060.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2026-02-21T22:04:10.002 INFO:teuthology.orchestra.run.trial060.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 533 kB/s | 42 kB 00:00 2026-02-21T22:04:10.209 INFO:teuthology.orchestra.run.trial060.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 25 MB/s | 5.3 MB 00:00 2026-02-21T22:04:10.397 INFO:teuthology.orchestra.run.trial060.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2026-02-21T22:04:10.479 INFO:teuthology.orchestra.run.trial060.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2026-02-21T22:04:10.561 INFO:teuthology.orchestra.run.trial060.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 3.0 MB/s | 249 kB 00:00 2026-02-21T22:04:10.641 INFO:teuthology.orchestra.run.trial060.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 597 kB/s | 48 kB 00:00 2026-02-21T22:04:10.722 INFO:teuthology.orchestra.run.trial060.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-21T22:04:10.810 INFO:teuthology.orchestra.run.trial060.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2026-02-21T22:04:10.890 INFO:teuthology.orchestra.run.trial060.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 436 kB/s | 35 kB 00:00 2026-02-21T22:04:11.078 INFO:teuthology.orchestra.run.trial060.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 33 MB/s | 6.1 MB 00:00 2026-02-21T22:04:11.161 INFO:teuthology.orchestra.run.trial060.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.2 MB/s | 442 kB 00:00 2026-02-21T22:04:11.241 INFO:teuthology.orchestra.run.trial060.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 961 kB/s | 77 kB 00:00 2026-02-21T22:04:11.321 INFO:teuthology.orchestra.run.trial060.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 3.2 MB/s | 267 kB 00:00 2026-02-21T22:04:11.403 INFO:teuthology.orchestra.run.trial060.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2026-02-21T22:04:11.485 INFO:teuthology.orchestra.run.trial060.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 3.3 MB/s | 279 kB 00:00 2026-02-21T22:04:11.565 INFO:teuthology.orchestra.run.trial060.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 669 kB/s | 54 kB 00:00 2026-02-21T22:04:12.186 INFO:teuthology.orchestra.run.trial060.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-21T22:04:12.266 INFO:teuthology.orchestra.run.trial060.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 521 kB/s | 42 kB 00:00 2026-02-21T22:04:12.348 INFO:teuthology.orchestra.run.trial060.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-21T22:04:12.427 INFO:teuthology.orchestra.run.trial060.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-21T22:04:12.456 INFO:teuthology.orchestra.run.trial060.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 764 kB/s | 22 kB 00:00 2026-02-21T22:04:12.500 INFO:teuthology.orchestra.run.trial060.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 19 MB/s | 862 kB 00:00 2026-02-21T22:04:12.777 INFO:teuthology.orchestra.run.trial060.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.9 MB/s | 551 kB 00:00 2026-02-21T22:04:12.813 INFO:teuthology.orchestra.run.trial060.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 543 kB/s | 19 kB 00:00 2026-02-21T22:04:12.855 INFO:teuthology.orchestra.run.trial060.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2026-02-21T22:04:12.916 INFO:teuthology.orchestra.run.trial060.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 8.9 MB/s | 548 kB 00:00 2026-02-21T22:04:12.954 INFO:teuthology.orchestra.run.trial060.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 770 kB/s | 29 kB 00:00 2026-02-21T22:04:12.992 INFO:teuthology.orchestra.run.trial060.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 1.6 MB/s | 60 kB 00:00 2026-02-21T22:04:13.030 INFO:teuthology.orchestra.run.trial060.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.1 MB/s | 43 kB 00:00 2026-02-21T22:04:13.066 INFO:teuthology.orchestra.run.trial060.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 874 kB/s | 32 kB 00:00 2026-02-21T22:04:13.103 INFO:teuthology.orchestra.run.trial060.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 385 kB/s | 14 kB 00:00 2026-02-21T22:04:13.145 INFO:teuthology.orchestra.run.trial060.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 4.1 MB/s | 173 kB 00:00 2026-02-21T22:04:13.194 INFO:teuthology.orchestra.run.trial060.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 7.2 MB/s | 358 kB 00:00 2026-02-21T22:04:13.238 INFO:teuthology.orchestra.run.trial060.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 5.6 MB/s | 254 kB 00:00 2026-02-21T22:04:13.318 INFO:teuthology.orchestra.run.trial060.stdout:(69/96): ceph-mgr-diskprediction-local-20.3.0-5 1.2 MB/s | 7.4 MB 00:05 2026-02-21T22:04:13.349 INFO:teuthology.orchestra.run.trial060.stdout:(70/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 18 MB/s | 2.0 MB 00:00 2026-02-21T22:04:13.387 INFO:teuthology.orchestra.run.trial060.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 3.5 MB/s | 139 kB 00:00 2026-02-21T22:04:13.426 INFO:teuthology.orchestra.run.trial060.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.4 MB/s | 56 kB 00:00 2026-02-21T22:04:13.462 INFO:teuthology.orchestra.run.trial060.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 295 kB/s | 11 kB 00:00 2026-02-21T22:04:13.499 INFO:teuthology.orchestra.run.trial060.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 482 kB/s | 18 kB 00:00 2026-02-21T22:04:13.518 INFO:teuthology.orchestra.run.trial060.stdout:(75/96): python3-grpcio-tools-1.46.7-10.el9.x86 725 kB/s | 144 kB 00:00 2026-02-21T22:04:13.535 INFO:teuthology.orchestra.run.trial060.stdout:(76/96): python3-jaraco-collections-3.0.0-8.el9 632 kB/s | 23 kB 00:00 2026-02-21T22:04:13.556 INFO:teuthology.orchestra.run.trial060.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 507 kB/s | 20 kB 00:00 2026-02-21T22:04:13.572 INFO:teuthology.orchestra.run.trial060.stdout:(78/96): python3-jaraco-functools-3.5.0-2.el9.n 530 kB/s | 19 kB 00:00 2026-02-21T22:04:13.597 INFO:teuthology.orchestra.run.trial060.stdout:(79/96): python3-jaraco-text-4.0.0-2.el9.noarch 656 kB/s | 26 kB 00:00 2026-02-21T22:04:13.656 INFO:teuthology.orchestra.run.trial060.stdout:(80/96): python3-kubernetes-26.1.0-3.el9.noarch 12 MB/s | 1.0 MB 00:00 2026-02-21T22:04:13.658 INFO:teuthology.orchestra.run.trial060.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2026-02-21T22:04:13.694 INFO:teuthology.orchestra.run.trial060.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.2 MB/s | 86 kB 00:00 2026-02-21T22:04:13.704 INFO:teuthology.orchestra.run.trial060.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.2 MB/s | 58 kB 00:00 2026-02-21T22:04:13.731 INFO:teuthology.orchestra.run.trial060.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 454 kB/s | 16 kB 00:00 2026-02-21T22:04:13.755 INFO:teuthology.orchestra.run.trial060.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2026-02-21T22:04:13.767 INFO:teuthology.orchestra.run.trial060.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 842 kB/s | 31 kB 00:00 2026-02-21T22:04:13.804 INFO:teuthology.orchestra.run.trial060.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 1.6 MB/s | 59 kB 00:00 2026-02-21T22:04:13.823 INFO:teuthology.orchestra.run.trial060.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2026-02-21T22:04:13.842 INFO:teuthology.orchestra.run.trial060.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 3.2 MB/s | 125 kB 00:00 2026-02-21T22:04:13.864 INFO:teuthology.orchestra.run.trial060.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 894 kB/s | 36 kB 00:00 2026-02-21T22:04:13.879 INFO:teuthology.orchestra.run.trial060.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 2.3 MB/s | 86 kB 00:00 2026-02-21T22:04:13.910 INFO:teuthology.orchestra.run.trial060.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 1.9 MB/s | 90 kB 00:00 2026-02-21T22:04:13.915 INFO:teuthology.orchestra.run.trial060.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.3 MB/s | 48 kB 00:00 2026-02-21T22:04:13.948 INFO:teuthology.orchestra.run.trial060.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 577 kB/s | 22 kB 00:00 2026-02-21T22:04:13.952 INFO:teuthology.orchestra.run.trial060.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 551 kB/s | 20 kB 00:00 2026-02-21T22:04:17.972 INFO:teuthology.orchestra.run.trial060.stdout:(96/96): ceph-mgr-dashboard-20.3.0-5425.g7c8cf9 4.7 MB/s | 51 MB 00:10 2026-02-21T22:04:17.974 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:17.974 INFO:teuthology.orchestra.run.trial060.stdout:Total 11 MB/s | 158 MB 00:14 2026-02-21T22:04:18.385 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:18.423 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:18.423 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:19.022 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:19.023 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:19.664 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:19.666 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-21T22:04:19.670 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-21T22:04:19.691 INFO:teuthology.orchestra.run.trial060.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-21T22:04:19.725 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-21T22:04:19.731 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-21T22:04:19.734 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-21T22:04:19.738 INFO:teuthology.orchestra.run.trial060.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-21T22:04:19.761 INFO:teuthology.orchestra.run.trial060.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-21T22:04:19.774 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-21T22:04:19.778 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-21T22:04:19.784 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-21T22:04:19.785 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-21T22:04:19.797 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-21T22:04:19.803 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-21T22:04:19.827 INFO:teuthology.orchestra.run.trial060.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-21T22:04:19.830 INFO:teuthology.orchestra.run.trial060.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-21T22:04:19.879 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-21T22:04:19.892 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-21T22:04:19.895 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-21T22:04:19.899 INFO:teuthology.orchestra.run.trial060.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-21T22:04:19.904 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-21T22:04:19.909 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-21T22:04:19.912 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-21T22:04:19.926 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-21T22:04:19.930 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-21T22:04:19.936 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-21T22:04:19.947 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-21T22:04:19.956 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-21T22:04:19.962 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-21T22:04:19.981 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-21T22:04:19.993 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-21T22:04:20.008 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-21T22:04:20.012 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-21T22:04:20.020 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-21T22:04:20.060 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-21T22:04:20.382 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-21T22:04:20.391 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-21T22:04:20.395 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-21T22:04:20.401 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-21T22:04:20.403 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-21T22:04:20.430 INFO:teuthology.orchestra.run.trial060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-21T22:04:20.473 INFO:teuthology.orchestra.run.trial060.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-21T22:04:20.482 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-21T22:04:20.487 INFO:teuthology.orchestra.run.trial060.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-21T22:04:20.491 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-21T22:04:20.497 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-21T22:04:20.500 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-21T22:04:20.507 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-21T22:04:20.511 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-21T22:04:20.538 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-21T22:04:20.548 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-21T22:04:20.554 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-21T22:04:20.560 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-21T22:04:20.595 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-21T22:04:20.817 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-21T22:04:20.840 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-21T22:04:20.844 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-21T22:04:20.903 INFO:teuthology.orchestra.run.trial060.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-21T22:04:20.905 INFO:teuthology.orchestra.run.trial060.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-21T22:04:20.926 INFO:teuthology.orchestra.run.trial060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-21T22:04:21.251 INFO:teuthology.orchestra.run.trial060.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-21T22:04:21.323 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-21T22:04:21.990 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-21T22:04:22.006 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-21T22:04:22.016 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-21T22:04:22.022 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-21T22:04:22.036 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-21T22:04:22.051 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-21T22:04:22.126 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-21T22:04:22.135 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-21T22:04:22.158 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-21T22:04:22.188 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-21T22:04:22.238 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-21T22:04:22.244 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-21T22:04:22.247 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-21T22:04:22.249 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-21T22:04:22.267 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T22:04:22.267 INFO:teuthology.orchestra.run.trial060.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-21T22:04:22.267 INFO:teuthology.orchestra.run.trial060.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-21T22:04:22.268 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:22.275 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T22:04:22.301 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T22:04:22.301 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-21T22:04:22.302 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:22.338 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-21T22:04:22.955 INFO:teuthology.orchestra.run.trial060.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-21T22:04:23.712 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_6 80/96 2026-02-21T22:04:23.776 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_6 81/96 2026-02-21T22:04:23.778 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 82/96 2026-02-21T22:04:23.793 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 82/96 2026-02-21T22:04:23.793 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:23.793 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-21T22:04:23.793 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:23.804 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 83/96 2026-02-21T22:04:23.829 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 83/96 2026-02-21T22:04:23.829 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:23.829 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-21T22:04:23.830 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T22:04:23.830 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T22:04:23.830 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:23.886 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 84/96 2026-02-21T22:04:23.888 INFO:teuthology.orchestra.run.trial060.stdout: Installing : cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 84/96 2026-02-21T22:04:23.897 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5425.g7c8cf961.el9 85/96 2026-02-21T22:04:23.921 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5425.g7c8cf961.el 86/96 2026-02-21T22:04:23.922 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarc 87/96 2026-02-21T22:04:26.813 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarc 87/96 2026-02-21T22:04:26.820 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noa 88/96 2026-02-21T22:04:27.273 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noa 88/96 2026-02-21T22:04:27.275 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf 89/96 2026-02-21T22:04:27.287 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf 89/96 2026-02-21T22:04:27.288 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noa 90/96 2026-02-21T22:04:27.351 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noa 90/96 2026-02-21T22:04:27.393 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5425.g7c8cf961.el9. 91/96 2026-02-21T22:04:27.400 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 92/96 2026-02-21T22:04:27.415 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 92/96 2026-02-21T22:04:27.415 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:27.415 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-21T22:04:27.416 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T22:04:27.416 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T22:04:27.416 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:27.428 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 93/96 2026-02-21T22:04:27.652 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 93/96 2026-02-21T22:04:27.653 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 94/96 2026-02-21T22:04:27.675 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 94/96 2026-02-21T22:04:27.675 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:27.675 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-21T22:04:27.675 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T22:04:27.675 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T22:04:27.676 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:27.769 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 95/96 2026-02-21T22:04:27.790 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 95/96 2026-02-21T22:04:27.790 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:27.790 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-21T22:04:27.790 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T22:04:27.790 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T22:04:27.790 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:27.791 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 96/96 2026-02-21T22:04:27.798 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_6 96/96 2026-02-21T22:04:27.823 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_6 96/96 2026-02-21T22:04:28.643 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 96/96 2026-02-21T22:04:28.643 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/96 2026-02-21T22:04:28.643 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/96 2026-02-21T22:04:28.643 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 4/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 5/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_6 6/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_6 7/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5425.g7c8cf961.el 8/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarc 9/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noa 10/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf 11/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noa 12/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5425.g7c8cf961.el9. 13/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 14/96 2026-02-21T22:04:28.644 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5425.g7c8cf961.el9 15/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 16/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 17/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-21T22:04:28.645 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-21T22:04:28.646 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-21T22:04:28.647 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-21T22:04:28.648 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-21T22:04:28.649 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-21T22:04:28.650 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-21T22:04:28.651 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-21T22:04:28.651 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-21T22:04:28.651 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-21T22:04:28.651 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout: ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout: ceph-grafana-dashboards-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.706 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-modules-core-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:28.707 INFO:teuthology.orchestra.run.trial060.stdout: ceph-prometheus-alerts-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-21T22:04:28.708 INFO:teuthology.orchestra.run.trial060.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: lua-5.4.4-4.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-21T22:04:28.709 INFO:teuthology.orchestra.run.trial060.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-21T22:04:28.710 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-21T22:04:28.711 INFO:teuthology.orchestra.run.trial060.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-ply-3.11-14.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-21T22:04:28.712 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-21T22:04:28.713 INFO:teuthology.orchestra.run.trial060.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-21T22:04:28.714 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-21T22:04:28.715 INFO:teuthology.orchestra.run.trial060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-21T22:04:28.715 INFO:teuthology.orchestra.run.trial060.stdout: unzip-6.0-59.el9.x86_64 2026-02-21T22:04:28.715 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-21T22:04:28.715 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-21T22:04:28.715 INFO:teuthology.orchestra.run.trial060.stdout: zip-3.0-35.el9.x86_64 2026-02-21T22:04:28.715 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:28.716 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:28.767 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-base 2026-02-21T22:04:29.080 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:29.143 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:29.162 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:29.163 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:29.163 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:29.184 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install cephadm 2026-02-21T22:04:29.496 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:29.560 INFO:teuthology.orchestra.run.trial060.stdout:Package cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T22:04:29.578 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:29.579 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:29.579 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:29.600 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T22:04:29.910 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:29.993 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:29.993 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:29.993 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-21T22:04:29.993 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:29.993 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 147 k 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 147 k 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 419 k 2026-02-21T22:04:29.994 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:30.412 INFO:teuthology.orchestra.run.trial060.stdout:ceph-immutable-object-cache-20.3.0-5425.g7c8cf9 352 kB/s | 147 kB 00:00 2026-02-21T22:04:30.412 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:30.413 INFO:teuthology.orchestra.run.trial060.stdout:Total 351 kB/s | 147 kB 00:00 2026-02-21T22:04:30.413 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:30.417 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:30.417 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:30.449 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:30.449 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:30.504 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:30.505 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961. 1/1 2026-02-21T22:04:30.528 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961. 1/1 2026-02-21T22:04:30.528 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:30.528 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-21T22:04:30.528 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:30.825 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961. 1/1 2026-02-21T22:04:30.825 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:30.825 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:30.825 INFO:teuthology.orchestra.run.trial060.stdout: ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:30.825 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:30.826 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:30.853 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr 2026-02-21T22:04:31.162 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:31.226 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:31.244 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:31.245 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:31.245 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:31.266 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T22:04:31.576 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:31.639 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T22:04:31.658 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:31.658 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:31.658 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:31.680 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T22:04:31.985 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:32.049 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T22:04:32.067 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:32.068 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:32.068 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:32.088 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-rook 2026-02-21T22:04:32.401 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:00 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:32.467 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T22:04:32.485 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:32.486 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:32.486 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:32.507 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T22:04:32.816 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:00 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:32.881 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T22:04:32.899 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:32.900 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:32.900 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:32.922 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-osd 2026-02-21T22:04:33.233 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:01 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:33.297 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:33.315 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:33.316 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:33.316 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:33.337 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-osd-classic 2026-02-21T22:04:33.647 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:01 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:33.714 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:33.733 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:33.733 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:33.733 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:33.755 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-fuse 2026-02-21T22:04:34.062 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:02 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:34.145 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout: ceph-fuse x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 987 k 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:34.146 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:34.147 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:34.147 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 987 k 2026-02-21T22:04:34.147 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 2.9 M 2026-02-21T22:04:34.147 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:34.779 INFO:teuthology.orchestra.run.trial060.stdout:ceph-fuse-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 1.5 MB/s | 987 kB 00:00 2026-02-21T22:04:34.780 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:34.780 INFO:teuthology.orchestra.run.trial060.stdout:Total 1.5 MB/s | 987 kB 00:00 2026-02-21T22:04:34.780 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:34.784 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:34.785 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:34.819 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:34.819 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:34.911 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:34.928 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:35.161 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:35.217 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:35.218 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:35.218 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:35.218 INFO:teuthology.orchestra.run.trial060.stdout: ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:35.218 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:35.218 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:35.245 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-volume 2026-02-21T22:04:35.560 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:03 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:35.625 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T22:04:35.643 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:35.643 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:35.643 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:35.665 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install librados-devel 2026-02-21T22:04:35.973 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:03 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:36.056 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout: librados-devel x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 123 k 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 123 k 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 437 k 2026-02-21T22:04:36.057 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:36.521 INFO:teuthology.orchestra.run.trial060.stdout:librados-devel-20.3.0-5425.g7c8cf961.el9.x86_64 266 kB/s | 123 kB 00:00 2026-02-21T22:04:36.521 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:36.521 INFO:teuthology.orchestra.run.trial060.stdout:Total 265 kB/s | 123 kB 00:00 2026-02-21T22:04:36.521 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:36.524 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:36.525 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:36.557 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:36.557 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:36.612 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:36.629 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:36.713 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:36.770 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:36.771 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:36.771 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:36.771 INFO:teuthology.orchestra.run.trial060.stdout: librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:36.771 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:36.771 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:36.799 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install libcephfs2 2026-02-21T22:04:37.122 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:05 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:37.188 INFO:teuthology.orchestra.run.trial060.stdout:Package libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:37.206 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:37.207 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:37.207 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:37.229 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install libcephfs-devel 2026-02-21T22:04:37.544 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:05 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:37.627 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-devel x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 37 k 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 37 k 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 171 k 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:37.970 INFO:teuthology.orchestra.run.trial060.stdout:libcephfs-devel-20.3.0-5425.g7c8cf961.el9.x86_6 109 kB/s | 37 kB 00:00 2026-02-21T22:04:37.971 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:37.971 INFO:teuthology.orchestra.run.trial060.stdout:Total 109 kB/s | 37 kB 00:00 2026-02-21T22:04:37.971 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:37.973 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:37.973 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:37.976 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:37.976 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:37.998 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:38.014 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:38.069 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:38.126 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:38.126 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:38.126 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:38.126 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:38.126 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:38.126 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:38.153 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install librados2 2026-02-21T22:04:38.459 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:06 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:38.523 INFO:teuthology.orchestra.run.trial060.stdout:Package librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:38.541 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:38.542 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:38.542 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:38.562 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install librbd1 2026-02-21T22:04:38.872 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:06 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:38.936 INFO:teuthology.orchestra.run.trial060.stdout:Package librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:38.955 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:38.955 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:38.955 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:38.976 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-rados 2026-02-21T22:04:39.288 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:07 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:39.353 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:39.371 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:39.372 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:39.372 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:39.395 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-rgw 2026-02-21T22:04:39.706 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:07 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:39.773 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:39.791 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:39.792 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:39.792 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:39.814 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-cephfs 2026-02-21T22:04:40.131 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:08 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:40.197 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:40.215 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:40.216 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:40.216 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:40.238 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-rbd 2026-02-21T22:04:40.547 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:08 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:40.612 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T22:04:40.631 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:40.631 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-21T22:04:40.631 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:40.653 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install rbd-fuse 2026-02-21T22:04:40.963 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:08 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout: rbd-fuse x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 84 k 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:41.047 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:41.048 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:41.048 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:41.048 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:41.048 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 84 k 2026-02-21T22:04:41.048 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 209 k 2026-02-21T22:04:41.048 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:41.354 INFO:teuthology.orchestra.run.trial060.stdout:rbd-fuse-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 273 kB/s | 84 kB 00:00 2026-02-21T22:04:41.355 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:41.355 INFO:teuthology.orchestra.run.trial060.stdout:Total 272 kB/s | 84 kB 00:00 2026-02-21T22:04:41.355 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:41.359 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:41.359 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:41.391 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:41.391 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:41.441 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:41.458 INFO:teuthology.orchestra.run.trial060.stdout: Installing : rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:41.543 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:41.600 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:41.600 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:41.600 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:41.600 INFO:teuthology.orchestra.run.trial060.stdout: rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:41.600 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:41.600 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:41.629 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install rbd-mirror 2026-02-21T22:04:41.942 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:09 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:42.026 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:42.026 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout: rbd-mirror x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 2.9 M 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:42.027 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 2.9 M 2026-02-21T22:04:42.028 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 11 M 2026-02-21T22:04:42.028 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:43.265 INFO:teuthology.orchestra.run.trial060.stdout:rbd-mirror-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 2.3 MB/s | 2.9 MB 00:01 2026-02-21T22:04:43.266 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:43.266 INFO:teuthology.orchestra.run.trial060.stdout:Total 2.3 MB/s | 2.9 MB 00:01 2026-02-21T22:04:43.266 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:43.271 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:43.271 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:43.311 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:43.311 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:43.487 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:43.488 INFO:teuthology.orchestra.run.trial060.stdout: Installing : rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:43.513 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:43.513 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T22:04:43.513 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-21T22:04:43.513 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T22:04:43.513 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T22:04:43.513 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:43.843 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:43.843 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:43.843 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:43.844 INFO:teuthology.orchestra.run.trial060.stdout: rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:43.844 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:43.844 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:43.872 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install rbd-nbd 2026-02-21T22:04:44.183 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:12 ago on Sat 21 Feb 2026 10:03:32 PM UTC. 2026-02-21T22:04:44.266 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-21T22:04:44.267 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:44.267 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-21T22:04:44.267 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout: rbd-nbd x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 172 k 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 172 k 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 462 k 2026-02-21T22:04:44.268 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-21T22:04:44.731 INFO:teuthology.orchestra.run.trial060.stdout:rbd-nbd-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 372 kB/s | 172 kB 00:00 2026-02-21T22:04:44.732 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-21T22:04:44.733 INFO:teuthology.orchestra.run.trial060.stdout:Total 371 kB/s | 172 kB 00:00 2026-02-21T22:04:44.733 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-21T22:04:44.737 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-21T22:04:44.737 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-21T22:04:44.769 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-21T22:04:44.769 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-21T22:04:44.826 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-21T22:04:44.842 INFO:teuthology.orchestra.run.trial060.stdout: Installing : rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:44.925 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:44.984 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T22:04:44.984 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:44.984 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-21T22:04:44.984 INFO:teuthology.orchestra.run.trial060.stdout: rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T22:04:44.984 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:44.984 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-21T22:04:45.013 DEBUG:teuthology.parallel:result is None 2026-02-21T22:04:45.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:04:45.134 DEBUG:teuthology.orchestra.run.trial060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T22:04:45.155 INFO:teuthology.orchestra.run.trial060.stdout:20.3.0-5425.g7c8cf961.el9 2026-02-21T22:04:45.156 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5425.g7c8cf961.el9 2026-02-21T22:04:45.156 INFO:teuthology.task.install:The correct ceph version 20.3.0-5425.g7c8cf961 is installed. 2026-02-21T22:04:45.158 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T22:04:45.158 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:45.158 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T22:04:45.220 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T22:04:45.220 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:45.220 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T22:04:45.284 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T22:04:45.349 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T22:04:45.349 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:45.349 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T22:04:45.412 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T22:04:45.476 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T22:04:45.476 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:45.476 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T22:04:45.540 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T22:04:45.604 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T22:04:45.726 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T22:04:45.726 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 777 /var/log/ceph 2026-02-21T22:04:45.751 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T22:04:45.752 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T22:04:45.814 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T22:04:45.814 DEBUG:teuthology.orchestra.run.trial060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T22:04:45.884 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T22:04:45.884 INFO:tasks.ceph:config {'conf': {'osd': {'osd min pg log entries': 5, 'osd_fast_fail_on_connection_refused': False, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T22:04:45.884 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078', 'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle', 'description': 'rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore/{alloc$/{stupid} base mem$/{normal-1} onode-segment$/{1M} write$/{v1/{compr$/{yes$/{zlib}} v1}}}} rados supported-random-distro$/{centos_latest}}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '63078', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'distro-suffix': 'rockylinux-10'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '53f8441172c46e34523029ab88f9db7a45514e84'}}, 'owner': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 1998, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/NitzanMordhai/ceph.git', 'suite_sha1': '53f8441172c46e34523029ab88f9db7a45514e84', 'targets': {'trial060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5NQXVoY4h38zhNQ+yrKXz4nEI9dcd+t80hflziAP8JI6jej2Kq2lkI6rvza84qoGBqObQi3pvEIQAuyWr8rkM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd min pg log entries': 5, 'osd_fast_fail_on_connection_refused': False, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'cluster': 'ceph'}}, {'osd_recovery.test_incomplete_pgs': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-21_21:55:34', 'tube': 'trial', 'user': 'nmordech', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T22:04:45.884 DEBUG:teuthology.orchestra.run.trial060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T22:04:45.935 DEBUG:teuthology.orchestra.run.trial060:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T22:04:45.999 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:45.999 DEBUG:teuthology.orchestra.run.trial060:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T22:04:46.054 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T22:04:46.054 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_1 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1314 Links: 1 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-21 22:04:43.665087906 +0000 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-21 22:03:40.108170250 +0000 2026-02-21T22:04:46.110 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-21 22:03:40.108170250 +0000 2026-02-21T22:04:46.111 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-21 22:03:40.108170250 +0000 2026-02-21T22:04:46.111 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T22:04:46.172 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-21T22:04:46.172 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-21T22:04:46.172 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.0001503 s, 3.4 MB/s 2026-02-21T22:04:46.173 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T22:04:46.232 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_2 2026-02-21T22:04:46.289 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T22:04:46.289 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-21 22:04:43.665087906 +0000 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-21 22:03:40.107170289 +0000 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-21 22:03:40.107170289 +0000 2026-02-21T22:04:46.290 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-21 22:03:40.107170289 +0000 2026-02-21T22:04:46.291 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T22:04:46.354 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-21T22:04:46.354 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-21T22:04:46.354 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.000148037 s, 3.5 MB/s 2026-02-21T22:04:46.355 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T22:04:46.412 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_3 2026-02-21T22:04:46.469 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T22:04:46.469 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:04:46.469 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-21T22:04:46.469 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:04:46.470 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T22:04:46.470 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-21 22:04:43.666087897 +0000 2026-02-21T22:04:46.470 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-21 22:03:40.107170289 +0000 2026-02-21T22:04:46.470 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-21 22:03:40.107170289 +0000 2026-02-21T22:04:46.470 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-21 22:03:40.107170289 +0000 2026-02-21T22:04:46.470 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T22:04:46.533 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-21T22:04:46.533 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-21T22:04:46.533 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.000158375 s, 3.2 MB/s 2026-02-21T22:04:46.534 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T22:04:46.590 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_4 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1319 Links: 1 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-21 22:04:43.666087897 +0000 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-21 22:03:40.108170250 +0000 2026-02-21T22:04:46.648 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-21 22:03:40.108170250 +0000 2026-02-21T22:04:46.649 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-21 22:03:40.108170250 +0000 2026-02-21T22:04:46.649 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T22:04:46.713 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-21T22:04:46.713 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-21T22:04:46.713 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.000161451 s, 3.2 MB/s 2026-02-21T22:04:46.714 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T22:04:46.771 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T22:04:46.772 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-21T22:04:46.772 INFO:tasks.ceph:Generating config... 2026-02-21T22:04:46.772 INFO:tasks.ceph:[osd] osd min pg log entries = 5 2026-02-21T22:04:46.772 INFO:tasks.ceph:[osd] osd_fast_fail_on_connection_refused = False 2026-02-21T22:04:46.772 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore onode segment size = 1024K 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore write v2 = False 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] debug bluefs = 20 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] debug bluestore = 20 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd op num shards = 1 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd op num threads per shard = 2 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-21T22:04:46.773 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-21T22:04:46.773 INFO:tasks.ceph:[global] mon election default strategy = 1 2026-02-21T22:04:46.773 INFO:tasks.ceph:[global] ms type = async 2026-02-21T22:04:46.773 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T22:04:46.773 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T22:04:46.773 INFO:tasks.ceph:[mon] debug mon = 30 2026-02-21T22:04:46.773 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-21T22:04:46.773 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-21T22:04:46.773 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-21T22:04:46.773 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T22:04:46.774 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T22:04:46.844 INFO:teuthology.orchestra.run.trial060.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T22:04:46.847 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T22:04:46.921 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T22:04:46.984 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.60'), ('mon.b', '[v2:10.20.193.60:3301,v1:10.20.193.60:6790]'), ('mon.c', '[v2:10.20.193.60:3302,v1:10.20.193.60:6791]')] 2026-02-21T22:04:46.985 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.60,[v2:10.20.193.60:3301,v1:10.20.193.60:6790],[v2:10.20.193.60:3302,v1:10.20.193.60:6791]', 'mon election default strategy': 1, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd min pg log entries': 5, 'osd_fast_fail_on_connection_refused': False, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 30, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-21T22:04:46.987 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:46.987 DEBUG:teuthology.orchestra.run.trial060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T22:04:47.040 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.60 --addv b '[v2:10.20.193.60:3301,v1:10.20.193.60:6790]' --addv c '[v2:10.20.193.60:3302,v1:10.20.193.60:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:47.113 INFO:teuthology.orchestra.run.trial060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:47.113 INFO:teuthology.orchestra.run.trial060.stdout:monmaptool: generated fsid ab2fd705-c792-40f6-9c14-b3ae92aaa530 2026-02-21T22:04:47.113 INFO:teuthology.orchestra.run.trial060.stdout:setting min_mon_release = tentacle 2026-02-21T22:04:47.113 INFO:teuthology.orchestra.run.trial060.stdout:epoch 0 2026-02-21T22:04:47.113 INFO:teuthology.orchestra.run.trial060.stdout:fsid ab2fd705-c792-40f6-9c14-b3ae92aaa530 2026-02-21T22:04:47.113 INFO:teuthology.orchestra.run.trial060.stdout:last_changed 2026-02-21T22:04:47.111098+0000 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:created 2026-02-21T22:04:47.111098+0000 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:min_mon_release 20 (tentacle) 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:election_strategy: 1 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:0: [v2:10.20.193.60:3300/0,v1:10.20.193.60:6789/0] mon.a 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:1: [v2:10.20.193.60:3301/0,v1:10.20.193.60:6790/0] mon.b 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:2: [v2:10.20.193.60:3302/0,v1:10.20.193.60:6791/0] mon.c 2026-02-21T22:04:47.114 INFO:teuthology.orchestra.run.trial060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-21T22:04:47.116 DEBUG:teuthology.orchestra.run.trial060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T22:04:47.173 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ab2fd705-c792-40f6-9c14-b3ae92aaa530... 2026-02-21T22:04:47.175 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T22:04:47.253 INFO:teuthology.orchestra.run.trial060.stdout:[global] 2026-02-21T22:04:47.253 INFO:teuthology.orchestra.run.trial060.stdout: chdir = "" 2026-02-21T22:04:47.253 INFO:teuthology.orchestra.run.trial060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T22:04:47.253 INFO:teuthology.orchestra.run.trial060.stdout: auth supported = cephx 2026-02-21T22:04:47.253 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: filestore xattr use omap = true 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon clock drift allowed = 1.000 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: osd crush chooseleaf type = 0 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: auth debug = true 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: ms die on old message = true 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: ms die on bug = true 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon pg warn max object skew = 0 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: osd pool default size = 2 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon osd allow primary affinity = true 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon osd allow pg remap = true 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on legacy crush tunables = false 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on crush straw calc version zero = false 2026-02-21T22:04:47.254 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on no sortbitwise = false 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on osd down out interval zero = false 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on too few osds = false 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon_allow_pool_size_one = true 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: osd default data pool replay window = 5 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon allow pool delete = true 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: mon cluster log file level = debug 2026-02-21T22:04:47.255 INFO:teuthology.orchestra.run.trial060.stdout: debug asserts on shutdown = true 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: mon health detail to clog = false 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: mon host = "10.20.193.60,[v2:10.20.193.60:3301,v1:10.20.193.60:6790],[v2:10.20.193.60:3302,v1:10.20.193.60:6791]" 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: mon election default strategy = 1 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: ms type = async 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: fsid = ab2fd705-c792-40f6-9c14-b3ae92aaa530 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout:[osd] 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: osd journal size = 100 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: osd scrub load threshold = 5.0 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: osd scrub max interval = 600 2026-02-21T22:04:47.256 INFO:teuthology.orchestra.run.trial060.stdout: osd mclock profile = high_recovery_ops 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd mclock skip benchmark = True 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd recover clone overlap = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd recovery max chunk = 1048576 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd debug shutdown = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd debug op order = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd debug verify stray on activate = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd debug trim objects = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd open classes on start = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: osd debug pg log writeout = true 2026-02-21T22:04:47.257 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: osd deep scrub update digest min age = 30 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: osd map max advance = 10 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: journal zero on create = true 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: filestore ondisk finisher threads = 3 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: filestore apply finisher threads = 3 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: bdev debug aio = true 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: osd debug misdirected ops = true 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: osd min pg log entries = 5 2026-02-21T22:04:47.258 INFO:teuthology.orchestra.run.trial060.stdout: osd_fast_fail_on_connection_refused = False 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bdev async discard = True 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bdev enable discard = True 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore allocator = stupid 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore block size = 96636764160 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore compression algorithm = zlib 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore compression mode = aggressive 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore fsck on mount = True 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore onode segment size = 1024K 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore write v2 = False 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: bluestore zero block detection = True 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: debug bluefs = 20 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: debug bluestore = 20 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: debug ms = 1 2026-02-21T22:04:47.259 INFO:teuthology.orchestra.run.trial060.stdout: debug osd = 20 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: debug rocksdb = 10 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: mon osd backfillfull_ratio = 0.85 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: mon osd full ratio = 0.9 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: mon osd nearfull ratio = 0.8 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd debug verify cached snaps = True 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd debug verify missing on start = True 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd failsafe full ratio = 0.95 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd mclock override recovery settings = True 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd objectstore = bluestore 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd op num shards = 1 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd op num threads per shard = 2 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd op queue = debug_random 2026-02-21T22:04:47.260 INFO:teuthology.orchestra.run.trial060.stdout: osd op queue cut off = debug_random 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout:[mgr] 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: debug ms = 1 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: debug mgr = 20 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: debug mon = 20 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: debug auth = 20 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min pgs per osd = 4 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min bytes per osd = 10 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: mgr/telemetry/nag = false 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout:[mon] 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: debug ms = 1 2026-02-21T22:04:47.261 INFO:teuthology.orchestra.run.trial060.stdout: debug mon = 30 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: debug paxos = 20 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: debug auth = 20 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: mon data avail warn = 5 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: mon mgr mkfs grace = 240 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min pgs per osd = 4 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: mon osd reporter subtree level = osd 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: mon osd prime pg temp = true 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min bytes per osd = 10 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T22:04:47.262 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: # 1m isn't quite enough 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: mon_down_mkfs_grace = 2m 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_filestore_osds = false 2026-02-21T22:04:47.263 INFO:teuthology.orchestra.run.trial060.stdout: mon scrub interval = 300 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout:[client] 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout: rgw cache enabled = true 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout: rgw enable ops log = true 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout: rgw enable usage log = true 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout:[mon.a] 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout:[mon.b] 2026-02-21T22:04:47.264 INFO:teuthology.orchestra.run.trial060.stdout:[mon.c] 2026-02-21T22:04:47.266 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T22:04:47.266 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T22:04:47.346 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T22:04:47.347 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:47.347 DEBUG:teuthology.orchestra.run.trial060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T22:04:47.405 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:47.405 DEBUG:teuthology.orchestra.run.trial060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T22:04:47.462 INFO:tasks.ceph:Sending monmap to node ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:04:47.462 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:47.462 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T22:04:47.462 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T22:04:47.534 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:47.534 DEBUG:teuthology.orchestra.run.trial060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:47.589 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T22:04:47.590 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T22:04:47.590 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T22:04:47.672 INFO:teuthology.orchestra.run.trial060.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T22:04:47.674 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T22:04:47.675 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T22:04:47.675 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T22:04:47.675 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-21T22:04:47.675 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T22:04:47.739 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T22:04:47.739 INFO:tasks.ceph:role: osd.0 2026-02-21T22:04:47.739 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:04:47.739 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:04:47.805 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:04:47.806 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:04:47.806 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:04:48.376 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-21T22:04:48.378 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-21T22:04:48.378 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T22:04:48.449 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T22:04:48.517 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T22:04:48.581 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T22:04:48.582 INFO:tasks.ceph:role: osd.1 2026-02-21T22:04:48.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:04:48.582 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T22:04:48.647 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:04:48.647 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:04:48.647 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:04:48.648 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:04:49.205 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-21T22:04:49.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-21T22:04:49.207 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T22:04:49.275 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T22:04:49.343 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T22:04:49.407 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T22:04:49.408 INFO:tasks.ceph:role: osd.2 2026-02-21T22:04:49.408 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:04:49.408 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:04:49.472 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:04:50.028 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-21T22:04:50.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-21T22:04:50.030 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T22:04:50.097 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T22:04:50.163 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-21T22:04:50.229 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T22:04:50.229 INFO:tasks.ceph:role: osd.3 2026-02-21T22:04:50.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:04:50.229 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:04:50.293 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:04:50.849 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-21T22:04:50.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-21T22:04:50.850 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2026-02-21T22:04:50.922 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-21T22:04:50.990 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:51.066 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.064+0000 7f2db6e35780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-21T22:04:51.067 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.064+0000 7f2db6e35780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-21T22:04:51.067 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.064+0000 7f2db6e35780 -1 bdev(0x2a8d400 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-21T22:04:51.067 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.064+0000 7f2db6e35780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-21T22:04:51.514 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T22:04:51.578 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:51.654 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.651+0000 7fbf3222f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-21T22:04:51.654 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.652+0000 7fbf3222f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-21T22:04:51.654 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.652+0000 7fbf3222f780 -1 bdev(0x3ba1400 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-21T22:04:51.655 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:51.652+0000 7fbf3222f780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-21T22:04:52.118 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T22:04:52.184 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:52.259 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.256+0000 7f575c635780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-21T22:04:52.259 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.257+0000 7f575c635780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-21T22:04:52.259 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.257+0000 7f575c635780 -1 bdev(0x41e9400 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-21T22:04:52.259 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.257+0000 7f575c635780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-21T22:04:52.730 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T22:04:52.795 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:52.872 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.869+0000 7ff2c1a40780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-21T22:04:52.872 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.870+0000 7ff2c1a40780 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-21T22:04:52.872 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.870+0000 7ff2c1a40780 -1 bdev(0x26b3400 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-21T22:04:52.872 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-21T22:04:52.870+0000 7ff2c1a40780 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-21T22:04:53.341 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-21T22:04:53.407 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T22:04:53.407 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:53.407 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T22:04:53.471 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:53.471 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T22:04:53.536 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:53.536 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T22:04:53.600 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:53.600 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T22:04:53.664 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:53.665 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-21T22:04:53.728 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T22:04:53.728 DEBUG:teuthology.orchestra.run.trial060:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T22:04:53.791 INFO:teuthology.orchestra.run.trial060.stdout:[mgr.x] 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout: key = AQD/K5pp2aXxJxAAEBrP0MxON+TkPqidMwCg4w== 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout:[osd.0] 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout: key = AQADLJppGQfdAxAA4z3ShS4Rmn8/iHOzIeLDNg== 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout:[osd.1] 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout: key = AQADLJppLP7lJhAAl3F1NnOco5TXHPvXxAZAnw== 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout:[osd.2] 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout: key = AQAELJppnI1XDxAAuKPV/vWuxB5Xt0qWASiiYg== 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout:[osd.3] 2026-02-21T22:04:53.792 INFO:teuthology.orchestra.run.trial060.stdout: key = AQAELJppSKvhMxAAyWyu7+y0i3DcbmrnC+v4nQ== 2026-02-21T22:04:53.794 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T22:04:53.872 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:04:53.951 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:04:54.031 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:04:54.110 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:04:54.187 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T22:04:54.187 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T22:04:54.251 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T22:04:54.331 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T22:04:54.395 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-21T22:04:54.459 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T22:04:54.540 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-21T22:04:54.604 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-21T22:04:54.670 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T22:04:54.751 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-21T22:04:54.816 DEBUG:teuthology.orchestra.run.trial060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:04:54.871 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T22:04:54.871 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T22:04:54.871 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T22:04:54.912 INFO:tasks.ceph.mon.a:Started 2026-02-21T22:04:54.912 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:04:54.912 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:04:54.914 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:04:54.914 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-21T22:04:54.914 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-21T22:04:54.915 INFO:tasks.ceph.mon.c:Started 2026-02-21T22:04:54.916 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph config set mgr mgr_pool false --force 2026-02-21T22:04:55.228 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T22:04:55.229 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T22:04:55.229 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T22:04:55.230 INFO:tasks.ceph.mgr.x:Started 2026-02-21T22:04:55.230 DEBUG:tasks.ceph:set 0 configs 2026-02-21T22:04:55.230 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph config dump 2026-02-21T22:04:55.294 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.290+0000 7f966ea24fc0 -1 mgr[py] Loading module stats in main interpreter 2026-02-21T22:04:55.325 INFO:teuthology.orchestra.run.trial060.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T22:04:55.325 INFO:teuthology.orchestra.run.trial060.stdout:mgr dev mgr_pool false * 2026-02-21T22:04:55.328 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.324+0000 7f966ea24fc0 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-21T22:04:55.331 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T22:04:55.332 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T22:04:55.432 INFO:teuthology.orchestra.run.trial060.stderr:adjusted tunables profile to default 2026-02-21T22:04:55.440 INFO:tasks.ceph:check_enable_crimson: False 2026-02-21T22:04:55.440 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T22:04:55.440 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:55.440 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T22:04:55.464 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:55.464 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T22:04:55.528 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:55.528 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T22:04:55.594 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-21T22:04:55.594 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-21T22:04:55.622 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.618+0000 7f966ea24fc0 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-21T22:04:55.630 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.626+0000 7f966ea24fc0 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-21T22:04:55.636 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.632+0000 7f966ea24fc0 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-21T22:04:55.641 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.638+0000 7f966ea24fc0 -1 mgr[py] Loading module balancer in main interpreter 2026-02-21T22:04:55.647 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new 171d695f-2570-47af-aab5-ae0bd86f3717 0 2026-02-21T22:04:55.647 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.643+0000 7f966ea24fc0 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-21T22:04:55.659 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.655+0000 7f966ea24fc0 -1 mgr[py] Loading module smb in main interpreter 2026-02-21T22:04:55.680 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.677+0000 7f966ea24fc0 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-21T22:04:55.683 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.679+0000 7f966ea24fc0 -1 mgr[py] Loading module alerts in main interpreter 2026-02-21T22:04:55.684 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.681+0000 7f966ea24fc0 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-21T22:04:55.690 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.687+0000 7f966ea24fc0 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-21T22:04:55.726 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.722+0000 7f966ea24fc0 -1 mgr[py] Loading module status in main interpreter 2026-02-21T22:04:55.727 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.724+0000 7f966ea24fc0 -1 mgr[py] Loading module rgw in main interpreter 2026-02-21T22:04:55.731 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.728+0000 7f966ea24fc0 -1 mgr[py] Loading module insights in main interpreter 2026-02-21T22:04:55.733 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.729+0000 7f966ea24fc0 -1 mgr[py] Loading module iostat in main interpreter 2026-02-21T22:04:55.734 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.730+0000 7f966ea24fc0 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-21T22:04:55.748 INFO:teuthology.orchestra.run.trial060.stdout:0 2026-02-21T22:04:55.752 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.748+0000 7f966ea24fc0 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-21T22:04:55.753 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.749+0000 7f966ea24fc0 -1 mgr[py] Loading module crash in main interpreter 2026-02-21T22:04:55.755 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.751+0000 7f966ea24fc0 -1 mgr[py] Loading module progress in main interpreter 2026-02-21T22:04:55.758 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new 638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9 1 2026-02-21T22:04:55.758 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.754+0000 7f966ea24fc0 -1 mgr[py] Loading module rook in main interpreter 2026-02-21T22:04:55.872 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.868+0000 7f966ea24fc0 -1 mgr[py] Loading module nfs in main interpreter 2026-02-21T22:04:55.872 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.868+0000 7f966ea24fc0 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-21T22:04:55.874 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.870+0000 7f966ea24fc0 -1 mgr[py] Loading module selftest in main interpreter 2026-02-21T22:04:55.876 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.873+0000 7f966ea24fc0 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-21T22:04:55.878 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.874+0000 7f966ea24fc0 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-21T22:04:55.878 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.875+0000 7f966ea24fc0 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-21T22:04:55.878 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:55.875+0000 7f966ea24fc0 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-21T22:04:55.908 INFO:teuthology.orchestra.run.trial060.stdout:1 2026-02-21T22:04:55.916 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new fd94ed65-fa21-4566-aa25-5e039e1b7d32 2 2026-02-21T22:04:56.020 INFO:teuthology.orchestra.run.trial060.stdout:2 2026-02-21T22:04:56.029 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new b5d2feef-3a3b-48bd-a117-8263471172c5 3 2026-02-21T22:04:56.055 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:56.051+0000 7f966ea24fc0 -1 mgr[py] Loading module volumes in main interpreter 2026-02-21T22:04:56.080 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:56.076+0000 7f966ea24fc0 -1 mgr[py] Loading module k8sevents in main interpreter 2026-02-21T22:04:56.082 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:56.079+0000 7f966ea24fc0 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-21T22:04:56.083 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:56.080+0000 7f966ea24fc0 -1 mgr[py] Loading module localpool in main interpreter 2026-02-21T22:04:56.084 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:56.080+0000 7f966ea24fc0 -1 mgr[py] Loading module influx in main interpreter 2026-02-21T22:04:56.134 INFO:teuthology.orchestra.run.trial060.stdout:3 2026-02-21T22:04:56.142 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T22:04:56.143 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-21T22:04:56.185 INFO:tasks.ceph.osd.0:Started 2026-02-21T22:04:56.185 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T22:04:56.185 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-21T22:04:56.188 INFO:tasks.ceph.osd.1:Started 2026-02-21T22:04:56.188 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T22:04:56.188 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-21T22:04:56.190 INFO:tasks.ceph.osd.2:Started 2026-02-21T22:04:56.190 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-21T22:04:56.191 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-21T22:04:56.193 INFO:tasks.ceph.osd.3:Started 2026-02-21T22:04:56.193 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:04:56.293 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:04:56.294 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":6,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:04:56.130176+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T22:04:56.130176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:04:56.301 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T22:04:56.301 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T22:04:56.426 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:56.422+0000 7f72b8c25780 -1 Falling back to public interface 2026-02-21T22:04:56.427 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:04:56.422+0000 7f0f99c10780 -1 Falling back to public interface 2026-02-21T22:04:56.427 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:04:56.422+0000 7f7a7a442780 -1 Falling back to public interface 2026-02-21T22:04:56.428 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:56.422+0000 7f72b6ffd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2026-02-21T22:04:56.428 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:04:56.422+0000 7f7a787fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2026-02-21T22:04:56.428 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:56.422+0000 7f72b6ffd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2026-02-21T22:04:56.429 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:04:56.422+0000 7f7a787fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2026-02-21T22:04:56.429 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:56.422+0000 7f72b6ffd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2026-02-21T22:04:56.429 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:56.423+0000 7f72b6ffd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6807/0: (98) Address already in use 2026-02-21T22:04:56.430 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-21T22:04:56.427+0000 7f2b9913d780 -1 Falling back to public interface 2026-02-21T22:04:56.703 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T22:04:56.793 INFO:teuthology.misc.health.trial060.stdout: 2026-02-21T22:04:56.793 INFO:teuthology.misc.health.trial060.stdout:{"epoch":6,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:04:56.130176+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T22:04:56.130176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:04:56.799 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-21T22:04:56.829 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:04:56.826+0000 7f0f99c10780 -1 osd.2 0 log_to_monitors true 2026-02-21T22:04:56.848 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:56.845+0000 7f72b8c25780 -1 osd.0 0 log_to_monitors true 2026-02-21T22:04:56.848 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-21T22:04:56.845+0000 7f2b9913d780 -1 osd.1 0 log_to_monitors true 2026-02-21T22:04:56.857 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:04:56.854+0000 7f7a7a442780 -1 osd.3 0 log_to_monitors true 2026-02-21T22:04:57.104 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:57.100+0000 7f959742b640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-21T22:04:57.104 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-21T22:04:57.100+0000 7f959742b640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-21T22:04:58.191 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-21T22:04:58.187+0000 7f2b94df7640 -1 osd.1 0 waiting for initial osdmap 2026-02-21T22:04:58.192 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:04:58.188+0000 7f0f94ff7640 -1 osd.2 0 waiting for initial osdmap 2026-02-21T22:04:58.192 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:58.188+0000 7f72b3ff7640 -1 osd.0 0 waiting for initial osdmap 2026-02-21T22:04:58.192 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:04:58.188+0000 7f7a757f7640 -1 osd.3 0 waiting for initial osdmap 2026-02-21T22:04:58.194 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-21T22:04:58.190+0000 7f72af7ee640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:04:58.195 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:04:58.191+0000 7f7a70fee640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:04:58.196 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:04:58.192+0000 7f0f90731640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:04:58.196 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-21T22:04:58.192+0000 7f2b90531640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:05:03.201 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T22:05:03.368 INFO:teuthology.misc.health.trial060.stdout: 2026-02-21T22:05:03.368 INFO:teuthology.misc.health.trial060.stdout:{"epoch":9,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:04:59.191916+0000","last_up_change":"2026-02-21T22:04:59.191916+0000","last_in_change":"2026-02-21T22:04:56.130176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6805","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6808","nonce":1304182596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6811","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6814","nonce":1304182596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6823","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6825","nonce":1304182596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6817","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6820","nonce":1304182596}]},"public_addr":"10.20.193.60:6808/1304182596","cluster_addr":"10.20.193.60:6814/1304182596","heartbeat_back_addr":"10.20.193.60:6825/1304182596","heartbeat_front_addr":"10.20.193.60:6820/1304182596","state":["exists","up"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6826","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6827","nonce":2501220781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6828","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6829","nonce":2501220781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6832","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6833","nonce":2501220781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6830","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6831","nonce":2501220781}]},"public_addr":"10.20.193.60:6827/2501220781","cluster_addr":"10.20.193.60:6829/2501220781","heartbeat_back_addr":"10.20.193.60:6833/2501220781","heartbeat_front_addr":"10.20.193.60:6831/2501220781","state":["exists","up"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6803","nonce":3447624660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6809","nonce":3447624660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6821","nonce":3447624660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6815","nonce":3447624660}]},"public_addr":"10.20.193.60:6803/3447624660","cluster_addr":"10.20.193.60:6809/3447624660","heartbeat_back_addr":"10.20.193.60:6821/3447624660","heartbeat_front_addr":"10.20.193.60:6815/3447624660","state":["exists","up"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6807","nonce":3833405627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6813","nonce":3833405627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6824","nonce":3833405627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6819","nonce":3833405627}]},"public_addr":"10.20.193.60:6807/3833405627","cluster_addr":"10.20.193.60:6813/3833405627","heartbeat_back_addr":"10.20.193.60:6824/3833405627","heartbeat_front_addr":"10.20.193.60:6819/3833405627","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:05:03.375 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-21T22:05:03.376 INFO:tasks.ceph:Creating RBD pool 2026-02-21T22:05:03.376 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T22:05:04.202 INFO:teuthology.orchestra.run.trial060.stderr:pool 'rbd' created 2026-02-21T22:05:04.213 DEBUG:teuthology.orchestra.run.trial060:> rbd --cluster ceph pool init rbd 2026-02-21T22:05:07.208 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T22:05:07.209 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T22:05:07.209 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T22:05:07.388 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:07.397 INFO:teuthology.orchestra.run.trial060.stdout:[{"version":2,"timestamp":"2026-02-21T22:04:55.217064+0000","name":"","changes":[{"name":"mgr/mgr_pool","new_value":"false"}]}] 2026-02-21T22:05:07.397 INFO:tasks.ceph_manager:config epoch is 2 2026-02-21T22:05:07.397 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T22:05:07.397 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T22:05:07.397 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T22:05:07.605 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:07.614 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":4,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6800","nonce":570277598},{"type":"v1","addr":"10.20.193.60:6801","nonce":570277598}]},"active_addr":"10.20.193.60:6801/570277598","active_change":"2026-02-21T22:04:56.091776+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.60:0","nonce":3009267616}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.60:0","nonce":2117321558}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.60:0","nonce":866172850}]}]} 2026-02-21T22:05:07.617 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T22:05:07.617 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T22:05:07.618 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:05:07.793 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:07.793 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":13,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:05:07.199682+0000","last_up_change":"2026-02-21T22:04:59.191916+0000","last_in_change":"2026-02-21T22:04:56.130176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T22:05:03.548523+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6805","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6808","nonce":1304182596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6811","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6814","nonce":1304182596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6823","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6825","nonce":1304182596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6817","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6820","nonce":1304182596}]},"public_addr":"10.20.193.60:6808/1304182596","cluster_addr":"10.20.193.60:6814/1304182596","heartbeat_back_addr":"10.20.193.60:6825/1304182596","heartbeat_front_addr":"10.20.193.60:6820/1304182596","state":["exists","up"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6826","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6827","nonce":2501220781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6828","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6829","nonce":2501220781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6832","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6833","nonce":2501220781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6830","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6831","nonce":2501220781}]},"public_addr":"10.20.193.60:6827/2501220781","cluster_addr":"10.20.193.60:6829/2501220781","heartbeat_back_addr":"10.20.193.60:6833/2501220781","heartbeat_front_addr":"10.20.193.60:6831/2501220781","state":["exists","up"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6803","nonce":3447624660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6809","nonce":3447624660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6821","nonce":3447624660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6815","nonce":3447624660}]},"public_addr":"10.20.193.60:6803/3447624660","cluster_addr":"10.20.193.60:6809/3447624660","heartbeat_back_addr":"10.20.193.60:6821/3447624660","heartbeat_front_addr":"10.20.193.60:6815/3447624660","state":["exists","up"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6807","nonce":3833405627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6813","nonce":3833405627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6824","nonce":3833405627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6819","nonce":3833405627}]},"public_addr":"10.20.193.60:6807/3833405627","cluster_addr":"10.20.193.60:6813/3833405627","heartbeat_back_addr":"10.20.193.60:6824/3833405627","heartbeat_front_addr":"10.20.193.60:6819/3833405627","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:05:07.802 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T22:05:07.802 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:05:07.978 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:07.978 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":13,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:05:07.199682+0000","last_up_change":"2026-02-21T22:04:59.191916+0000","last_in_change":"2026-02-21T22:04:56.130176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T22:05:03.548523+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6805","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6808","nonce":1304182596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6811","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6814","nonce":1304182596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6823","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6825","nonce":1304182596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6817","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6820","nonce":1304182596}]},"public_addr":"10.20.193.60:6808/1304182596","cluster_addr":"10.20.193.60:6814/1304182596","heartbeat_back_addr":"10.20.193.60:6825/1304182596","heartbeat_front_addr":"10.20.193.60:6820/1304182596","state":["exists","up"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6826","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6827","nonce":2501220781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6828","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6829","nonce":2501220781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6832","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6833","nonce":2501220781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6830","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6831","nonce":2501220781}]},"public_addr":"10.20.193.60:6827/2501220781","cluster_addr":"10.20.193.60:6829/2501220781","heartbeat_back_addr":"10.20.193.60:6833/2501220781","heartbeat_front_addr":"10.20.193.60:6831/2501220781","state":["exists","up"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6803","nonce":3447624660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6809","nonce":3447624660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6821","nonce":3447624660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6815","nonce":3447624660}]},"public_addr":"10.20.193.60:6803/3447624660","cluster_addr":"10.20.193.60:6809/3447624660","heartbeat_back_addr":"10.20.193.60:6821/3447624660","heartbeat_front_addr":"10.20.193.60:6815/3447624660","state":["exists","up"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6807","nonce":3833405627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6813","nonce":3833405627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6824","nonce":3833405627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6819","nonce":3833405627}]},"public_addr":"10.20.193.60:6807/3833405627","cluster_addr":"10.20.193.60:6813/3833405627","heartbeat_back_addr":"10.20.193.60:6824/3833405627","heartbeat_front_addr":"10.20.193.60:6819/3833405627","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:05:07.988 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T22:05:07.988 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T22:05:07.989 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T22:05:07.989 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T22:05:08.077 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:08.077 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:05:08.078 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-21T22:05:08.079 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:05:08.080 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-21T22:05:08.080 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:05:08.081 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:08.082 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:05:08.297 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:08.298 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-21T22:05:08.299 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:08.301 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-21T22:05:08.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2026-02-21T22:05:08.309 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:08.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.3 2026-02-21T22:05:08.310 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:08.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2026-02-21T22:05:08.311 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:08.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2026-02-21T22:05:08.312 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:08.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T22:05:08.313 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:08.527 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:08.528 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:08.535 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-21T22:05:08.096259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108296,"kb_used_data":456,"kb_used_omap":26,"kb_used_meta":107749,"kb_avail":377379064,"statfs":{"total":386547056640,"available":386436161536,"internally_reserved":0,"allocated":466944,"data_stored":125866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26648,"internal_metadata":110335976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201563+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:07.201563+0000","last_peered":"2026-02-21T22:05:07.201563+0000","last_clean":"2026-02-21T22:05:07.201563+0000","last_became_active":"2026-02-21T22:05:05.201756+0000","last_became_peered":"2026-02-21T22:05:05.201756+0000","last_unstale":"2026-02-21T22:05:07.201563+0000","last_undegraded":"2026-02-21T22:05:07.201563+0000","last_fullsized":"2026-02-21T22:05:07.201563+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:52:00.308603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029171399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201362+0000","last_change":"2026-02-21T22:05:07.201464+0000","last_active":"2026-02-21T22:05:07.201362+0000","last_peered":"2026-02-21T22:05:07.201362+0000","last_clean":"2026-02-21T22:05:07.201362+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:07.201362+0000","last_undegraded":"2026-02-21T22:05:07.201362+0000","last_fullsized":"2026-02-21T22:05:07.201362+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:44:25.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.330165+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:07.330165+0000","last_peered":"2026-02-21T22:05:07.330165+0000","last_clean":"2026-02-21T22:05:07.330165+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:07.330165+0000","last_undegraded":"2026-02-21T22:05:07.330165+0000","last_fullsized":"2026-02-21T22:05:07.330165+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201382+0000","last_change":"2026-02-21T22:05:07.201510+0000","last_active":"2026-02-21T22:05:07.201382+0000","last_peered":"2026-02-21T22:05:07.201382+0000","last_clean":"2026-02-21T22:05:07.201382+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:07.201382+0000","last_undegraded":"2026-02-21T22:05:07.201382+0000","last_fullsized":"2026-02-21T22:05:07.201382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:46:02.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201599+0000","last_change":"2026-02-21T22:05:07.201878+0000","last_active":"2026-02-21T22:05:07.201599+0000","last_peered":"2026-02-21T22:05:07.201599+0000","last_clean":"2026-02-21T22:05:07.201599+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:07.201599+0000","last_undegraded":"2026-02-21T22:05:07.201599+0000","last_fullsized":"2026-02-21T22:05:07.201599+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:35:06.806018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041015499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201695+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:07.201695+0000","last_peered":"2026-02-21T22:05:07.201695+0000","last_clean":"2026-02-21T22:05:07.201695+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:07.201695+0000","last_undegraded":"2026-02-21T22:05:07.201695+0000","last_fullsized":"2026-02-21T22:05:07.201695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:17:22.385745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201491+0000","last_change":"2026-02-21T22:05:07.201576+0000","last_active":"2026-02-21T22:05:07.201491+0000","last_peered":"2026-02-21T22:05:07.201491+0000","last_clean":"2026-02-21T22:05:07.201491+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:07.201491+0000","last_undegraded":"2026-02-21T22:05:07.201491+0000","last_fullsized":"2026-02-21T22:05:07.201491+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:59:25.312540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201736+0000","last_change":"2026-02-21T22:05:07.201865+0000","last_active":"2026-02-21T22:05:07.201736+0000","last_peered":"2026-02-21T22:05:07.201736+0000","last_clean":"2026-02-21T22:05:07.201736+0000","last_became_active":"2026-02-21T22:05:05.201979+0000","last_became_peered":"2026-02-21T22:05:05.201979+0000","last_unstale":"2026-02-21T22:05:07.201736+0000","last_undegraded":"2026-02-21T22:05:07.201736+0000","last_fullsized":"2026-02-21T22:05:07.201736+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:59:39.004753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025576700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6175,"internal_metadata":27584481},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:08.537 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:08.711 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:08.711 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:08.719 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-21T22:05:08.096259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108296,"kb_used_data":456,"kb_used_omap":26,"kb_used_meta":107749,"kb_avail":377379064,"statfs":{"total":386547056640,"available":386436161536,"internally_reserved":0,"allocated":466944,"data_stored":125866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26648,"internal_metadata":110335976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201563+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:07.201563+0000","last_peered":"2026-02-21T22:05:07.201563+0000","last_clean":"2026-02-21T22:05:07.201563+0000","last_became_active":"2026-02-21T22:05:05.201756+0000","last_became_peered":"2026-02-21T22:05:05.201756+0000","last_unstale":"2026-02-21T22:05:07.201563+0000","last_undegraded":"2026-02-21T22:05:07.201563+0000","last_fullsized":"2026-02-21T22:05:07.201563+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:52:00.308603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029171399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201362+0000","last_change":"2026-02-21T22:05:07.201464+0000","last_active":"2026-02-21T22:05:07.201362+0000","last_peered":"2026-02-21T22:05:07.201362+0000","last_clean":"2026-02-21T22:05:07.201362+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:07.201362+0000","last_undegraded":"2026-02-21T22:05:07.201362+0000","last_fullsized":"2026-02-21T22:05:07.201362+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:44:25.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.330165+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:07.330165+0000","last_peered":"2026-02-21T22:05:07.330165+0000","last_clean":"2026-02-21T22:05:07.330165+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:07.330165+0000","last_undegraded":"2026-02-21T22:05:07.330165+0000","last_fullsized":"2026-02-21T22:05:07.330165+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201382+0000","last_change":"2026-02-21T22:05:07.201510+0000","last_active":"2026-02-21T22:05:07.201382+0000","last_peered":"2026-02-21T22:05:07.201382+0000","last_clean":"2026-02-21T22:05:07.201382+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:07.201382+0000","last_undegraded":"2026-02-21T22:05:07.201382+0000","last_fullsized":"2026-02-21T22:05:07.201382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:46:02.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201599+0000","last_change":"2026-02-21T22:05:07.201878+0000","last_active":"2026-02-21T22:05:07.201599+0000","last_peered":"2026-02-21T22:05:07.201599+0000","last_clean":"2026-02-21T22:05:07.201599+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:07.201599+0000","last_undegraded":"2026-02-21T22:05:07.201599+0000","last_fullsized":"2026-02-21T22:05:07.201599+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:35:06.806018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041015499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201695+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:07.201695+0000","last_peered":"2026-02-21T22:05:07.201695+0000","last_clean":"2026-02-21T22:05:07.201695+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:07.201695+0000","last_undegraded":"2026-02-21T22:05:07.201695+0000","last_fullsized":"2026-02-21T22:05:07.201695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:17:22.385745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201491+0000","last_change":"2026-02-21T22:05:07.201576+0000","last_active":"2026-02-21T22:05:07.201491+0000","last_peered":"2026-02-21T22:05:07.201491+0000","last_clean":"2026-02-21T22:05:07.201491+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:07.201491+0000","last_undegraded":"2026-02-21T22:05:07.201491+0000","last_fullsized":"2026-02-21T22:05:07.201491+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:59:25.312540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-21T22:05:07.201736+0000","last_change":"2026-02-21T22:05:07.201865+0000","last_active":"2026-02-21T22:05:07.201736+0000","last_peered":"2026-02-21T22:05:07.201736+0000","last_clean":"2026-02-21T22:05:07.201736+0000","last_became_active":"2026-02-21T22:05:05.201979+0000","last_became_peered":"2026-02-21T22:05:05.201979+0000","last_unstale":"2026-02-21T22:05:07.201736+0000","last_undegraded":"2026-02-21T22:05:07.201736+0000","last_fullsized":"2026-02-21T22:05:07.201736+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:59:39.004753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025576700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6175,"internal_metadata":27584481},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:08.720 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T22:05:08.720 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T22:05:08.720 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T22:05:08.721 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T22:05:08.925 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:08.925 INFO:teuthology.orchestra.run.trial060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T22:05:08.933 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T22:05:08.934 INFO:teuthology.run_tasks:Running task osd_recovery.test_incomplete_pgs... 2026-02-21T22:05:08.958 INFO:tasks.osd_recovery:num_osds is 4 2026-02-21T22:05:08.959 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:05:09.134 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:09.135 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":13,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:05:07.199682+0000","last_up_change":"2026-02-21T22:04:59.191916+0000","last_in_change":"2026-02-21T22:04:56.130176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T22:05:03.548523+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6805","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6808","nonce":1304182596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6811","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6814","nonce":1304182596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6823","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6825","nonce":1304182596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6817","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6820","nonce":1304182596}]},"public_addr":"10.20.193.60:6808/1304182596","cluster_addr":"10.20.193.60:6814/1304182596","heartbeat_back_addr":"10.20.193.60:6825/1304182596","heartbeat_front_addr":"10.20.193.60:6820/1304182596","state":["exists","up"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6826","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6827","nonce":2501220781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6828","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6829","nonce":2501220781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6832","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6833","nonce":2501220781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6830","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6831","nonce":2501220781}]},"public_addr":"10.20.193.60:6827/2501220781","cluster_addr":"10.20.193.60:6829/2501220781","heartbeat_back_addr":"10.20.193.60:6833/2501220781","heartbeat_front_addr":"10.20.193.60:6831/2501220781","state":["exists","up"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6803","nonce":3447624660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6809","nonce":3447624660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6821","nonce":3447624660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":3447624660},{"type":"v1","addr":"10.20.193.60:6815","nonce":3447624660}]},"public_addr":"10.20.193.60:6803/3447624660","cluster_addr":"10.20.193.60:6809/3447624660","heartbeat_back_addr":"10.20.193.60:6821/3447624660","heartbeat_front_addr":"10.20.193.60:6815/3447624660","state":["exists","up"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6807","nonce":3833405627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6813","nonce":3833405627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6824","nonce":3833405627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":3833405627},{"type":"v1","addr":"10.20.193.60:6819","nonce":3833405627}]},"public_addr":"10.20.193.60:6807/3833405627","cluster_addr":"10.20.193.60:6813/3833405627","heartbeat_back_addr":"10.20.193.60:6824/3833405627","heartbeat_front_addr":"10.20.193.60:6819/3833405627","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:05:09.144 INFO:tasks.osd_recovery.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2026-02-21T22:05:03.548523+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 13, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-21T22:05:09.144 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2026-02-21T22:05:09.319 INFO:teuthology.orchestra.run.trial060.stdout:pg_num: 8 2026-02-21T22:05:09.327 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-21T22:05:09.505 INFO:teuthology.orchestra.run.trial060.stdout:epoch 14 2026-02-21T22:05:09.505 INFO:teuthology.orchestra.run.trial060.stdout:fsid ab2fd705-c792-40f6-9c14-b3ae92aaa530 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:created 2026-02-21T22:04:55.178617+0000 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:modified 2026-02-21T22:05:09.207347+0000 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:crush_version 4 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:full_ratio 0.95 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:backfillfull_ratio 0.9 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:nearfull_ratio 0.85 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:require_min_compat_client luminous 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:min_compat_client jewel 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:require_osd_release tentacle 2026-02-21T22:05:09.506 INFO:teuthology.orchestra.run.trial060.stdout:stretch_mode_enabled false 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.50 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout: removed_snaps_queue [2~1] 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout:max_osd 4 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout:osd.0 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6805/1304182596,v1:10.20.193.60:6808/1304182596] [v2:10.20.193.60:6811/1304182596,v1:10.20.193.60:6814/1304182596] exists,up 171d695f-2570-47af-aab5-ae0bd86f3717 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout:osd.1 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6826/2501220781,v1:10.20.193.60:6827/2501220781] [v2:10.20.193.60:6828/2501220781,v1:10.20.193.60:6829/2501220781] exists,up 638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout:osd.2 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6802/3447624660,v1:10.20.193.60:6803/3447624660] [v2:10.20.193.60:6806/3447624660,v1:10.20.193.60:6809/3447624660] exists,up fd94ed65-fa21-4566-aa25-5e039e1b7d32 2026-02-21T22:05:09.507 INFO:teuthology.orchestra.run.trial060.stdout:osd.3 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6804/3833405627,v1:10.20.193.60:6807/3833405627] [v2:10.20.193.60:6810/3833405627,v1:10.20.193.60:6813/3833405627] exists,up b5d2feef-3a3b-48bd-a117-8263471172c5 2026-02-21T22:05:09.514 INFO:tasks.osd_recovery.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6805/1304182596,v1:10.20.193.60:6808/1304182596] [v2:10.20.193.60:6811/1304182596,v1:10.20.193.60:6814/1304182596] exists,up 171d695f-2570-47af-aab5-ae0bd86f3717', 'osd.1 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6826/2501220781,v1:10.20.193.60:6827/2501220781] [v2:10.20.193.60:6828/2501220781,v1:10.20.193.60:6829/2501220781] exists,up 638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9', 'osd.2 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6802/3447624660,v1:10.20.193.60:6803/3447624660] [v2:10.20.193.60:6806/3447624660,v1:10.20.193.60:6809/3447624660] exists,up fd94ed65-fa21-4566-aa25-5e039e1b7d32', 'osd.3 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6804/3833405627,v1:10.20.193.60:6807/3833405627] [v2:10.20.193.60:6810/3833405627,v1:10.20.193.60:6813/3833405627] exists,up b5d2feef-3a3b-48bd-a117-8263471172c5'] 2026-02-21T22:05:09.514 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T22:05:09.515 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T22:05:09.516 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T22:05:09.516 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T22:05:09.605 INFO:teuthology.orchestra.run.trial060.stdout:38654705670 2026-02-21T22:05:09.605 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:05:09.607 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:09.607 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:05:09.608 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:09.609 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:05:09.610 INFO:teuthology.orchestra.run.trial060.stdout:38654705670 2026-02-21T22:05:09.611 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:05:09.788 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:09.800 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705670 got 38654705669 for osd.1 2026-02-21T22:05:09.831 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-21T22:05:09.831 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-21T22:05:09.831 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:09.841 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705668 for osd.3 2026-02-21T22:05:09.842 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705670 got 38654705669 for osd.2 2026-02-21T22:05:09.843 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705668 for osd.0 2026-02-21T22:05:10.800 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:05:10.842 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:05:10.843 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:05:10.844 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:05:10.982 INFO:teuthology.orchestra.run.trial060.stdout:38654705670 2026-02-21T22:05:10.993 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705670 got 38654705670 for osd.1 2026-02-21T22:05:10.993 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:11.022 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:11.022 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-21T22:05:11.025 INFO:teuthology.orchestra.run.trial060.stdout:38654705670 2026-02-21T22:05:11.033 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705669 for osd.0 2026-02-21T22:05:11.034 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:11.035 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705669 for osd.3 2026-02-21T22:05:11.035 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:11.036 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705670 got 38654705670 for osd.2 2026-02-21T22:05:11.036 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:11.037 INFO:tasks.osd_recovery.ceph_manager:waiting for clean 2026-02-21T22:05:11.037 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:11.251 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:11.252 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:11.261 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-21T22:05:10.096447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108296,"kb_used_data":456,"kb_used_omap":26,"kb_used_meta":107749,"kb_avail":377379064,"statfs":{"total":386547056640,"available":386436161536,"internally_reserved":0,"allocated":466944,"data_stored":125866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26648,"internal_metadata":110335976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000188"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209327+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:09.209327+0000","last_peered":"2026-02-21T22:05:09.209327+0000","last_clean":"2026-02-21T22:05:09.209327+0000","last_became_active":"2026-02-21T22:05:05.201756+0000","last_became_peered":"2026-02-21T22:05:05.201756+0000","last_unstale":"2026-02-21T22:05:09.209327+0000","last_undegraded":"2026-02-21T22:05:09.209327+0000","last_fullsized":"2026-02-21T22:05:09.209327+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:52:00.308603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029171399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209460+0000","last_change":"2026-02-21T22:05:07.201464+0000","last_active":"2026-02-21T22:05:09.209460+0000","last_peered":"2026-02-21T22:05:09.209460+0000","last_clean":"2026-02-21T22:05:09.209460+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:09.209460+0000","last_undegraded":"2026-02-21T22:05:09.209460+0000","last_fullsized":"2026-02-21T22:05:09.209460+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:44:25.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.530613+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:09.530613+0000","last_peered":"2026-02-21T22:05:09.530613+0000","last_clean":"2026-02-21T22:05:09.530613+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:09.530613+0000","last_undegraded":"2026-02-21T22:05:09.530613+0000","last_fullsized":"2026-02-21T22:05:09.530613+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209471+0000","last_change":"2026-02-21T22:05:07.201510+0000","last_active":"2026-02-21T22:05:09.209471+0000","last_peered":"2026-02-21T22:05:09.209471+0000","last_clean":"2026-02-21T22:05:09.209471+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:09.209471+0000","last_undegraded":"2026-02-21T22:05:09.209471+0000","last_fullsized":"2026-02-21T22:05:09.209471+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:46:02.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209365+0000","last_change":"2026-02-21T22:05:07.201878+0000","last_active":"2026-02-21T22:05:09.209365+0000","last_peered":"2026-02-21T22:05:09.209365+0000","last_clean":"2026-02-21T22:05:09.209365+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:09.209365+0000","last_undegraded":"2026-02-21T22:05:09.209365+0000","last_fullsized":"2026-02-21T22:05:09.209365+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:35:06.806018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041015499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":24,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209443+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:09.209443+0000","last_peered":"2026-02-21T22:05:09.209443+0000","last_clean":"2026-02-21T22:05:09.209443+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:09.209443+0000","last_undegraded":"2026-02-21T22:05:09.209443+0000","last_fullsized":"2026-02-21T22:05:09.209443+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:17:22.385745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209517+0000","last_change":"2026-02-21T22:05:07.201576+0000","last_active":"2026-02-21T22:05:09.209517+0000","last_peered":"2026-02-21T22:05:09.209517+0000","last_clean":"2026-02-21T22:05:09.209517+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:09.209517+0000","last_undegraded":"2026-02-21T22:05:09.209517+0000","last_fullsized":"2026-02-21T22:05:09.209517+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:59:25.312540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209378+0000","last_change":"2026-02-21T22:05:07.201865+0000","last_active":"2026-02-21T22:05:09.209378+0000","last_peered":"2026-02-21T22:05:09.209378+0000","last_clean":"2026-02-21T22:05:09.209378+0000","last_became_active":"2026-02-21T22:05:05.201979+0000","last_became_peered":"2026-02-21T22:05:05.201979+0000","last_unstale":"2026-02-21T22:05:09.209378+0000","last_undegraded":"2026-02-21T22:05:09.209378+0000","last_fullsized":"2026-02-21T22:05:09.209378+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:59:39.004753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025576700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6175,"internal_metadata":27584481},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:11.262 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:11.437 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:11.437 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:11.445 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-21T22:05:10.096447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108296,"kb_used_data":456,"kb_used_omap":26,"kb_used_meta":107749,"kb_avail":377379064,"statfs":{"total":386547056640,"available":386436161536,"internally_reserved":0,"allocated":466944,"data_stored":125866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26648,"internal_metadata":110335976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000188"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209327+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:09.209327+0000","last_peered":"2026-02-21T22:05:09.209327+0000","last_clean":"2026-02-21T22:05:09.209327+0000","last_became_active":"2026-02-21T22:05:05.201756+0000","last_became_peered":"2026-02-21T22:05:05.201756+0000","last_unstale":"2026-02-21T22:05:09.209327+0000","last_undegraded":"2026-02-21T22:05:09.209327+0000","last_fullsized":"2026-02-21T22:05:09.209327+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:52:00.308603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029171399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209460+0000","last_change":"2026-02-21T22:05:07.201464+0000","last_active":"2026-02-21T22:05:09.209460+0000","last_peered":"2026-02-21T22:05:09.209460+0000","last_clean":"2026-02-21T22:05:09.209460+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:09.209460+0000","last_undegraded":"2026-02-21T22:05:09.209460+0000","last_fullsized":"2026-02-21T22:05:09.209460+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:44:25.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.530613+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:09.530613+0000","last_peered":"2026-02-21T22:05:09.530613+0000","last_clean":"2026-02-21T22:05:09.530613+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:09.530613+0000","last_undegraded":"2026-02-21T22:05:09.530613+0000","last_fullsized":"2026-02-21T22:05:09.530613+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209471+0000","last_change":"2026-02-21T22:05:07.201510+0000","last_active":"2026-02-21T22:05:09.209471+0000","last_peered":"2026-02-21T22:05:09.209471+0000","last_clean":"2026-02-21T22:05:09.209471+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:09.209471+0000","last_undegraded":"2026-02-21T22:05:09.209471+0000","last_fullsized":"2026-02-21T22:05:09.209471+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:46:02.672085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209365+0000","last_change":"2026-02-21T22:05:07.201878+0000","last_active":"2026-02-21T22:05:09.209365+0000","last_peered":"2026-02-21T22:05:09.209365+0000","last_clean":"2026-02-21T22:05:09.209365+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:09.209365+0000","last_undegraded":"2026-02-21T22:05:09.209365+0000","last_fullsized":"2026-02-21T22:05:09.209365+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:35:06.806018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041015499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":24,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209443+0000","last_change":"2026-02-21T22:05:07.201695+0000","last_active":"2026-02-21T22:05:09.209443+0000","last_peered":"2026-02-21T22:05:09.209443+0000","last_clean":"2026-02-21T22:05:09.209443+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:09.209443+0000","last_undegraded":"2026-02-21T22:05:09.209443+0000","last_fullsized":"2026-02-21T22:05:09.209443+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:17:22.385745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209517+0000","last_change":"2026-02-21T22:05:07.201576+0000","last_active":"2026-02-21T22:05:09.209517+0000","last_peered":"2026-02-21T22:05:09.209517+0000","last_clean":"2026-02-21T22:05:09.209517+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:09.209517+0000","last_undegraded":"2026-02-21T22:05:09.209517+0000","last_fullsized":"2026-02-21T22:05:09.209517+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:59:25.312540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-21T22:05:09.209378+0000","last_change":"2026-02-21T22:05:07.201865+0000","last_active":"2026-02-21T22:05:09.209378+0000","last_peered":"2026-02-21T22:05:09.209378+0000","last_clean":"2026-02-21T22:05:09.209378+0000","last_became_active":"2026-02-21T22:05:05.201979+0000","last_became_peered":"2026-02-21T22:05:05.201979+0000","last_unstale":"2026-02-21T22:05:09.209378+0000","last_undegraded":"2026-02-21T22:05:09.209378+0000","last_fullsized":"2026-02-21T22:05:09.209378+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:59:39.004753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025576700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6175,"internal_metadata":27584481},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:11.446 INFO:tasks.osd_recovery.ceph_manager:clean! 2026-02-21T22:05:11.446 INFO:tasks.osd_recovery:Testing incomplete pgs... 2026-02-21T22:05:11.447 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set osd_recovery_delay_start 1000 2026-02-21T22:05:11.516 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:05:11.516 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_delay_start = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-21T22:05:11.516 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:05:11.521 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set osd_recovery_delay_start 1000 2026-02-21T22:05:11.589 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:05:11.589 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_delay_start = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-21T22:05:11.589 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:05:11.594 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set osd_recovery_delay_start 1000 2026-02-21T22:05:11.662 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:05:11.662 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_delay_start = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-21T22:05:11.663 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:05:11.668 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set osd_recovery_delay_start 1000 2026-02-21T22:05:11.734 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:05:11.734 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_delay_start = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-21T22:05:11.735 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:05:11.740 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 1 2026-02-21T22:05:11.920 INFO:tasks.ceph.mon.a.trial060.stderr:2026-02-21T22:05:11.916+0000 7fb3dbff1640 -1 mon.a@0(leader).osd e15 definitely_dead 0 2026-02-21T22:05:12.216 INFO:teuthology.orchestra.run.trial060.stderr:marked out osd.0. marked out osd.1. 2026-02-21T22:05:12.228 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T22:05:12.229 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T22:05:12.229 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T22:05:12.230 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T22:05:12.318 INFO:teuthology.orchestra.run.trial060.stdout:38654705671 2026-02-21T22:05:12.319 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:12.320 INFO:teuthology.orchestra.run.trial060.stdout:38654705672 2026-02-21T22:05:12.321 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:12.321 INFO:teuthology.orchestra.run.trial060.stdout:38654705672 2026-02-21T22:05:12.322 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:05:12.323 INFO:teuthology.orchestra.run.trial060.stdout:38654705671 2026-02-21T22:05:12.323 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:05:12.538 INFO:teuthology.orchestra.run.trial060.stdout:38654705671 2026-02-21T22:05:12.538 INFO:teuthology.orchestra.run.trial060.stdout:38654705670 2026-02-21T22:05:12.548 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705672 got 38654705671 for osd.2 2026-02-21T22:05:12.549 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705671 got 38654705670 for osd.3 2026-02-21T22:05:13.549 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:05:13.550 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:05:13.726 INFO:teuthology.orchestra.run.trial060.stdout:38654705671 2026-02-21T22:05:13.732 INFO:teuthology.orchestra.run.trial060.stdout:38654705670 2026-02-21T22:05:13.735 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705672 got 38654705671 for osd.2 2026-02-21T22:05:13.742 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705671 got 38654705670 for osd.3 2026-02-21T22:05:14.736 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:05:14.743 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:05:14.917 INFO:teuthology.orchestra.run.trial060.stdout:38654705672 2026-02-21T22:05:14.927 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705672 got 38654705672 for osd.2 2026-02-21T22:05:14.927 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:14.957 INFO:teuthology.orchestra.run.trial060.stdout:38654705671 2026-02-21T22:05:14.967 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705671 got 38654705671 for osd.3 2026-02-21T22:05:14.968 DEBUG:teuthology.parallel:result is None 2026-02-21T22:05:14.968 INFO:tasks.osd_recovery.ceph_manager:waiting for clean 2026-02-21T22:05:14.968 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:15.182 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:15.183 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:15.191 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-21T22:05:14.097013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54176,"kb_used_data":256,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689504,"statfs":{"total":193273528320,"available":193218052096,"internally_reserved":0,"allocated":262144,"data_stored":76210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.887918"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":4,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.215763+0000","last_change":"2026-02-21T22:05:12.215763+0000","last_active":"2026-02-21T22:05:12.215763+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:12.215763+0000","last_undegraded":"2026-02-21T22:05:12.215763+0000","last_fullsized":"2026-02-21T22:05:12.215763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972572+0000","last_change":"2026-02-21T22:05:12.215715+0000","last_active":"2026-02-21T22:05:12.215715+0000","last_peered":"2026-02-21T22:05:09.972572+0000","last_clean":"2026-02-21T22:05:09.972572+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:09.972572+0000","last_undegraded":"2026-02-21T22:05:09.972572+0000","last_fullsized":"2026-02-21T22:05:09.972572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-21T22:05:12.217052+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:12.217052+0000","last_peered":"2026-02-21T22:05:12.217052+0000","last_clean":"2026-02-21T22:05:12.217052+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:12.217052+0000","last_undegraded":"2026-02-21T22:05:12.217052+0000","last_fullsized":"2026-02-21T22:05:12.217052+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972574+0000","last_change":"2026-02-21T22:05:12.216218+0000","last_active":"2026-02-21T22:05:12.216218+0000","last_peered":"2026-02-21T22:05:09.972574+0000","last_clean":"2026-02-21T22:05:09.972574+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:09.972574+0000","last_undegraded":"2026-02-21T22:05:09.972574+0000","last_fullsized":"2026-02-21T22:05:09.972574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":8,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.216957+0000","last_change":"2026-02-21T22:05:12.216957+0000","last_active":"2026-02-21T22:05:12.216957+0000","last_peered":"2026-02-21T22:05:05.203891+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:12.216957+0000","last_undegraded":"2026-02-21T22:05:12.216957+0000","last_fullsized":"2026-02-21T22:05:12.216957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:09.972420+0000","last_change":"2026-02-21T22:05:12.310380+0000","last_active":"2026-02-21T22:05:12.310380+0000","last_peered":"2026-02-21T22:05:09.972420+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:09.972420+0000","last_undegraded":"2026-02-21T22:05:09.972420+0000","last_fullsized":"2026-02-21T22:05:09.972420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972686+0000","last_change":"2026-02-21T22:05:12.216374+0000","last_active":"2026-02-21T22:05:12.216374+0000","last_peered":"2026-02-21T22:05:09.972686+0000","last_clean":"2026-02-21T22:05:09.972686+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:09.972686+0000","last_undegraded":"2026-02-21T22:05:09.972686+0000","last_fullsized":"2026-02-21T22:05:09.972686+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:05.200482+0000","last_change":"2026-02-21T22:05:12.310911+0000","last_active":"2026-02-21T22:05:12.310911+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:05.200482+0000","last_undegraded":"2026-02-21T22:05:05.200482+0000","last_fullsized":"2026-02-21T22:05:05.200482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8202,"internal_metadata":27582454},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:15.192 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:15.367 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:15.368 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:15.375 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-21T22:05:14.097013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54176,"kb_used_data":256,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689504,"statfs":{"total":193273528320,"available":193218052096,"internally_reserved":0,"allocated":262144,"data_stored":76210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.887918"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":4,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.215763+0000","last_change":"2026-02-21T22:05:12.215763+0000","last_active":"2026-02-21T22:05:12.215763+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:12.215763+0000","last_undegraded":"2026-02-21T22:05:12.215763+0000","last_fullsized":"2026-02-21T22:05:12.215763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972572+0000","last_change":"2026-02-21T22:05:12.215715+0000","last_active":"2026-02-21T22:05:12.215715+0000","last_peered":"2026-02-21T22:05:09.972572+0000","last_clean":"2026-02-21T22:05:09.972572+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:09.972572+0000","last_undegraded":"2026-02-21T22:05:09.972572+0000","last_fullsized":"2026-02-21T22:05:09.972572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-21T22:05:12.217052+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:12.217052+0000","last_peered":"2026-02-21T22:05:12.217052+0000","last_clean":"2026-02-21T22:05:12.217052+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:12.217052+0000","last_undegraded":"2026-02-21T22:05:12.217052+0000","last_fullsized":"2026-02-21T22:05:12.217052+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972574+0000","last_change":"2026-02-21T22:05:12.216218+0000","last_active":"2026-02-21T22:05:12.216218+0000","last_peered":"2026-02-21T22:05:09.972574+0000","last_clean":"2026-02-21T22:05:09.972574+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:09.972574+0000","last_undegraded":"2026-02-21T22:05:09.972574+0000","last_fullsized":"2026-02-21T22:05:09.972574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":8,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.216957+0000","last_change":"2026-02-21T22:05:12.216957+0000","last_active":"2026-02-21T22:05:12.216957+0000","last_peered":"2026-02-21T22:05:05.203891+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:12.216957+0000","last_undegraded":"2026-02-21T22:05:12.216957+0000","last_fullsized":"2026-02-21T22:05:12.216957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:09.972420+0000","last_change":"2026-02-21T22:05:12.310380+0000","last_active":"2026-02-21T22:05:12.310380+0000","last_peered":"2026-02-21T22:05:09.972420+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:09.972420+0000","last_undegraded":"2026-02-21T22:05:09.972420+0000","last_fullsized":"2026-02-21T22:05:09.972420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972686+0000","last_change":"2026-02-21T22:05:12.216374+0000","last_active":"2026-02-21T22:05:12.216374+0000","last_peered":"2026-02-21T22:05:09.972686+0000","last_clean":"2026-02-21T22:05:09.972686+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:09.972686+0000","last_undegraded":"2026-02-21T22:05:09.972686+0000","last_fullsized":"2026-02-21T22:05:09.972686+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:05.200482+0000","last_change":"2026-02-21T22:05:12.310911+0000","last_active":"2026-02-21T22:05:12.310911+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:05.200482+0000","last_undegraded":"2026-02-21T22:05:05.200482+0000","last_fullsized":"2026-02-21T22:05:05.200482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8202,"internal_metadata":27582454},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:15.377 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:15.551 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:15.552 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:15.560 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-21T22:05:14.097013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54176,"kb_used_data":256,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689504,"statfs":{"total":193273528320,"available":193218052096,"internally_reserved":0,"allocated":262144,"data_stored":76210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.887918"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":4,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.215763+0000","last_change":"2026-02-21T22:05:12.215763+0000","last_active":"2026-02-21T22:05:12.215763+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:12.215763+0000","last_undegraded":"2026-02-21T22:05:12.215763+0000","last_fullsized":"2026-02-21T22:05:12.215763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972572+0000","last_change":"2026-02-21T22:05:12.215715+0000","last_active":"2026-02-21T22:05:12.215715+0000","last_peered":"2026-02-21T22:05:09.972572+0000","last_clean":"2026-02-21T22:05:09.972572+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:09.972572+0000","last_undegraded":"2026-02-21T22:05:09.972572+0000","last_fullsized":"2026-02-21T22:05:09.972572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-21T22:05:12.217052+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:12.217052+0000","last_peered":"2026-02-21T22:05:12.217052+0000","last_clean":"2026-02-21T22:05:12.217052+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:12.217052+0000","last_undegraded":"2026-02-21T22:05:12.217052+0000","last_fullsized":"2026-02-21T22:05:12.217052+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972574+0000","last_change":"2026-02-21T22:05:12.216218+0000","last_active":"2026-02-21T22:05:12.216218+0000","last_peered":"2026-02-21T22:05:09.972574+0000","last_clean":"2026-02-21T22:05:09.972574+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:09.972574+0000","last_undegraded":"2026-02-21T22:05:09.972574+0000","last_fullsized":"2026-02-21T22:05:09.972574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":8,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.216957+0000","last_change":"2026-02-21T22:05:12.216957+0000","last_active":"2026-02-21T22:05:12.216957+0000","last_peered":"2026-02-21T22:05:05.203891+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:12.216957+0000","last_undegraded":"2026-02-21T22:05:12.216957+0000","last_fullsized":"2026-02-21T22:05:12.216957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:09.972420+0000","last_change":"2026-02-21T22:05:12.310380+0000","last_active":"2026-02-21T22:05:12.310380+0000","last_peered":"2026-02-21T22:05:09.972420+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:09.972420+0000","last_undegraded":"2026-02-21T22:05:09.972420+0000","last_fullsized":"2026-02-21T22:05:09.972420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972686+0000","last_change":"2026-02-21T22:05:12.216374+0000","last_active":"2026-02-21T22:05:12.216374+0000","last_peered":"2026-02-21T22:05:09.972686+0000","last_clean":"2026-02-21T22:05:09.972686+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:09.972686+0000","last_undegraded":"2026-02-21T22:05:09.972686+0000","last_fullsized":"2026-02-21T22:05:09.972686+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:05.200482+0000","last_change":"2026-02-21T22:05:12.310911+0000","last_active":"2026-02-21T22:05:12.310911+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:05.200482+0000","last_undegraded":"2026-02-21T22:05:05.200482+0000","last_fullsized":"2026-02-21T22:05:05.200482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8202,"internal_metadata":27582454},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:15.561 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-21T22:05:15.561 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 16, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:12.215763+0000', 'last_change': '2026-02-21T22:05:12.215763+0000', 'last_active': '2026-02-21T22:05:12.215763+0000', 'last_peered': '2026-02-21T22:05:04.195147+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2026-02-21T22:05:12.215763+0000', 'last_undegraded': '2026-02-21T22:05:12.215763+0000', 'last_fullsized': '2026-02-21T22:05:12.215763+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:15.562 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:05:15.562 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "0'0", 'reported_seq': 25, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:09.972572+0000', 'last_change': '2026-02-21T22:05:12.215715+0000', 'last_active': '2026-02-21T22:05:12.215715+0000', 'last_peered': '2026-02-21T22:05:09.972572+0000', 'last_clean': '2026-02-21T22:05:09.972572+0000', 'last_became_active': '2026-02-21T22:05:05.201873+0000', 'last_became_peered': '2026-02-21T22:05:05.201873+0000', 'last_unstale': '2026-02-21T22:05:09.972572+0000', 'last_undegraded': '2026-02-21T22:05:09.972572+0000', 'last_fullsized': '2026-02-21T22:05:09.972572+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:15.562 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:05:15.562 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 25, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:09.972574+0000', 'last_change': '2026-02-21T22:05:12.216218+0000', 'last_active': '2026-02-21T22:05:12.216218+0000', 'last_peered': '2026-02-21T22:05:09.972574+0000', 'last_clean': '2026-02-21T22:05:09.972574+0000', 'last_became_active': '2026-02-21T22:05:05.201810+0000', 'last_became_peered': '2026-02-21T22:05:05.201810+0000', 'last_unstale': '2026-02-21T22:05:09.972574+0000', 'last_undegraded': '2026-02-21T22:05:09.972574+0000', 'last_fullsized': '2026-02-21T22:05:09.972574+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:15.562 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:15.562 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 8, 'reported_epoch': 16, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:12.216957+0000', 'last_change': '2026-02-21T22:05:12.216957+0000', 'last_active': '2026-02-21T22:05:12.216957+0000', 'last_peered': '2026-02-21T22:05:05.203891+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:05.203833+0000', 'last_became_peered': '2026-02-21T22:05:05.203833+0000', 'last_unstale': '2026-02-21T22:05:12.216957+0000', 'last_undegraded': '2026-02-21T22:05:12.216957+0000', 'last_fullsized': '2026-02-21T22:05:12.216957+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:15.563 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:15.563 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 2, 'reported_epoch': 16, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:09.972420+0000', 'last_change': '2026-02-21T22:05:12.310380+0000', 'last_active': '2026-02-21T22:05:12.310380+0000', 'last_peered': '2026-02-21T22:05:09.972420+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:05.202003+0000', 'last_became_peered': '2026-02-21T22:05:05.202003+0000', 'last_unstale': '2026-02-21T22:05:09.972420+0000', 'last_undegraded': '2026-02-21T22:05:09.972420+0000', 'last_fullsized': '2026-02-21T22:05:09.972420+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:15.563 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:05:15.563 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "0'0", 'reported_seq': 25, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:09.972686+0000', 'last_change': '2026-02-21T22:05:12.216374+0000', 'last_active': '2026-02-21T22:05:12.216374+0000', 'last_peered': '2026-02-21T22:05:09.972686+0000', 'last_clean': '2026-02-21T22:05:09.972686+0000', 'last_became_active': '2026-02-21T22:05:05.202070+0000', 'last_became_peered': '2026-02-21T22:05:05.202070+0000', 'last_unstale': '2026-02-21T22:05:09.972686+0000', 'last_undegraded': '2026-02-21T22:05:09.972686+0000', 'last_fullsized': '2026-02-21T22:05:09.972686+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:15.563 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-21T22:05:15.563 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 16, 'state': 'peering', 'last_fresh': '2026-02-21T22:05:05.200482+0000', 'last_change': '2026-02-21T22:05:12.310911+0000', 'last_active': '2026-02-21T22:05:12.310911+0000', 'last_peered': '2026-02-21T22:05:04.195147+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2026-02-21T22:05:05.200482+0000', 'last_undegraded': '2026-02-21T22:05:05.200482+0000', 'last_fullsized': '2026-02-21T22:05:05.200482+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:15.564 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:15.780 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:15.780 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":20,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":7},{"state_name":"active+clean","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55476224,"bytes_avail":193218052096,"bytes_total":193273528320,"inactive_pgs_ratio":0.875},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:15.789 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:15.789 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:15.963 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:15.964 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:15.972 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-21T22:05:14.097013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54176,"kb_used_data":256,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689504,"statfs":{"total":193273528320,"available":193218052096,"internally_reserved":0,"allocated":262144,"data_stored":76210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.887918"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":4,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.215763+0000","last_change":"2026-02-21T22:05:12.215763+0000","last_active":"2026-02-21T22:05:12.215763+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:12.215763+0000","last_undegraded":"2026-02-21T22:05:12.215763+0000","last_fullsized":"2026-02-21T22:05:12.215763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972572+0000","last_change":"2026-02-21T22:05:12.215715+0000","last_active":"2026-02-21T22:05:12.215715+0000","last_peered":"2026-02-21T22:05:09.972572+0000","last_clean":"2026-02-21T22:05:09.972572+0000","last_became_active":"2026-02-21T22:05:05.201873+0000","last_became_peered":"2026-02-21T22:05:05.201873+0000","last_unstale":"2026-02-21T22:05:09.972572+0000","last_undegraded":"2026-02-21T22:05:09.972572+0000","last_fullsized":"2026-02-21T22:05:09.972572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-21T22:05:12.217052+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:12.217052+0000","last_peered":"2026-02-21T22:05:12.217052+0000","last_clean":"2026-02-21T22:05:12.217052+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:12.217052+0000","last_undegraded":"2026-02-21T22:05:12.217052+0000","last_fullsized":"2026-02-21T22:05:12.217052+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972574+0000","last_change":"2026-02-21T22:05:12.216218+0000","last_active":"2026-02-21T22:05:12.216218+0000","last_peered":"2026-02-21T22:05:09.972574+0000","last_clean":"2026-02-21T22:05:09.972574+0000","last_became_active":"2026-02-21T22:05:05.201810+0000","last_became_peered":"2026-02-21T22:05:05.201810+0000","last_unstale":"2026-02-21T22:05:09.972574+0000","last_undegraded":"2026-02-21T22:05:09.972574+0000","last_fullsized":"2026-02-21T22:05:09.972574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":8,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:12.216957+0000","last_change":"2026-02-21T22:05:12.216957+0000","last_active":"2026-02-21T22:05:12.216957+0000","last_peered":"2026-02-21T22:05:05.203891+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:05.203833+0000","last_became_peered":"2026-02-21T22:05:05.203833+0000","last_unstale":"2026-02-21T22:05:12.216957+0000","last_undegraded":"2026-02-21T22:05:12.216957+0000","last_fullsized":"2026-02-21T22:05:12.216957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:09.972420+0000","last_change":"2026-02-21T22:05:12.310380+0000","last_active":"2026-02-21T22:05:12.310380+0000","last_peered":"2026-02-21T22:05:09.972420+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:05.202003+0000","last_became_peered":"2026-02-21T22:05:05.202003+0000","last_unstale":"2026-02-21T22:05:09.972420+0000","last_undegraded":"2026-02-21T22:05:09.972420+0000","last_fullsized":"2026-02-21T22:05:09.972420+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-21T22:05:09.972686+0000","last_change":"2026-02-21T22:05:12.216374+0000","last_active":"2026-02-21T22:05:12.216374+0000","last_peered":"2026-02-21T22:05:09.972686+0000","last_clean":"2026-02-21T22:05:09.972686+0000","last_became_active":"2026-02-21T22:05:05.202070+0000","last_became_peered":"2026-02-21T22:05:05.202070+0000","last_unstale":"2026-02-21T22:05:09.972686+0000","last_undegraded":"2026-02-21T22:05:09.972686+0000","last_fullsized":"2026-02-21T22:05:09.972686+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":16,"state":"peering","last_fresh":"2026-02-21T22:05:05.200482+0000","last_change":"2026-02-21T22:05:12.310911+0000","last_active":"2026-02-21T22:05:12.310911+0000","last_peered":"2026-02-21T22:05:04.195147+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-21T22:05:05.200482+0000","last_undegraded":"2026-02-21T22:05:05.200482+0000","last_fullsized":"2026-02-21T22:05:05.200482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":128,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":131072,"data_stored":38105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8202,"internal_metadata":27582454},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:18.974 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:19.153 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:19.154 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:19.162 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-21T22:05:18.097402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000954"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:19.163 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:19.338 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:19.339 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:19.347 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-21T22:05:18.097402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000954"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:19.348 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:19.348 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:19.348 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:19.349 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:19.349 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:19.561 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:19.562 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":24,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:19.570 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:19.570 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:19.744 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:19.744 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:19.752 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-21T22:05:18.097402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000954"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:22.754 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:22.931 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:22.932 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:22.939 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-21T22:05:22.097821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.883254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:22.941 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:23.116 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:23.117 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:23.125 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-21T22:05:22.097821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.883254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:23.126 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:23.127 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:23.127 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:23.127 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:23.127 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:23.345 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:23.346 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:23.355 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:23.355 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:23.529 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:23.529 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:23.537 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-21T22:05:22.097821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.883254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:26.539 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:26.718 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:26.718 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:26.727 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-21T22:05:26.098243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001229"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:26.729 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:26.905 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:26.905 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:26.914 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-21T22:05:26.098243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001229"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:26.915 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:26.915 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:26.915 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:26.915 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:26.916 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:27.132 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:27.132 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:27.141 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:27.141 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:27.317 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:27.318 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:27.326 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-21T22:05:26.098243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001229"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:30.328 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:30.509 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:30.509 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:30.518 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2026-02-21T22:05:30.098793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:30.519 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:30.695 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:30.695 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:30.704 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2026-02-21T22:05:30.098793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:30.705 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:30.705 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:30.705 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:30.705 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:30.706 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:30.924 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:30.924 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":35,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:30.933 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:30.933 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:31.112 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:31.113 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:31.121 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2026-02-21T22:05:30.098793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:34.123 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:34.302 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:34.302 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:34.311 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2026-02-21T22:05:34.099375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001553"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:34.312 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:34.487 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:34.487 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:34.496 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2026-02-21T22:05:34.099375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001553"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:34.498 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:34.498 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:34.498 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:34.498 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:34.498 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:34.712 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:34.713 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":39,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:34.722 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:34.722 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:34.899 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:34.900 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:34.908 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2026-02-21T22:05:34.099375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001553"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:37.910 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:38.097 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:38.098 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:38.107 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2026-02-21T22:05:36.099670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001698"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:38.109 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:38.290 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:38.290 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:38.299 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2026-02-21T22:05:38.099978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001735"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:38.301 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:38.301 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:38.301 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:38.301 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:38.302 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:38.518 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:38.518 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":43,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:38.528 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:38.528 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:38.710 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:38.711 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:38.719 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2026-02-21T22:05:38.099978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001735"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:41.721 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:41.902 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:41.903 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:41.911 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-21T22:05:40.100215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001693"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:41.912 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:42.086 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:42.087 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:42.095 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-21T22:05:40.100215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001693"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:42.096 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:42.096 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:42.096 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:42.097 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:42.097 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:42.311 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:42.311 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":47,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:42.321 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:42.322 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:42.498 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:42.499 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:42.506 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2026-02-21T22:05:42.100532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001738"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:45.509 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:45.688 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:45.689 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:45.697 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-21T22:05:44.100780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001729"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:45.699 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:45.875 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:45.876 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:45.884 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-21T22:05:44.100780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001729"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:45.885 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:45.885 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:45.886 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:45.886 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:45.886 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:46.104 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:46.105 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":50,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:46.115 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:46.115 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:46.291 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:46.291 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:46.301 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2026-02-21T22:05:46.101127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001751"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:49.303 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:49.483 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:49.483 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:49.491 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-21T22:05:48.101387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001716"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:49.493 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:49.669 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:49.670 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:49.678 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-21T22:05:48.101387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001716"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:49.679 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:49.679 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:49.680 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:49.680 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:49.680 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:49.898 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:49.898 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":54,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:49.907 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:49.907 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:50.082 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:50.083 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:50.092 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-21T22:05:48.101387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001716"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:53.094 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:53.272 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:53.272 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:53.281 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2026-02-21T22:05:52.101922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001706"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:53.282 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:53.456 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:53.457 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:53.465 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2026-02-21T22:05:52.101922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001706"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:53.466 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:53.466 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:53.466 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:53.466 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:53.467 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:53.684 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:53.685 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":58,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:53.693 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:53.694 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:53.873 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:53.873 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:53.881 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2026-02-21T22:05:52.101922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001706"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:56.883 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:57.060 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:57.061 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:57.068 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2026-02-21T22:05:56.102463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001682"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:57.070 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:57.248 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:57.249 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:57.257 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2026-02-21T22:05:56.102463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001682"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:05:57.259 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:05:57.259 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:05:57.259 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:05:57.259 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:05:57.259 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:05:57.477 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:57.477 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":62,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-21T22:04:55.179010+0000","services":{}},"progress_events":{}} 2026-02-21T22:05:57.486 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:05:57.486 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:05:57.661 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:05:57.661 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:05:57.669 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2026-02-21T22:05:56.102463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001682"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:00.670 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:00.849 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:00.850 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:00.858 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2026-02-21T22:06:00.102971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001583"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:00.859 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:01.033 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:01.034 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:01.042 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2026-02-21T22:06:00.102971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001583"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:01.043 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:01.043 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:01.043 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:01.044 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:01.044 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:01.263 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:01.264 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":66,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":2,"modified":"2026-02-21T22:05:58.102747+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-21T22:06:01.273 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:01.273 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:01.456 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:01.456 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:01.465 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2026-02-21T22:06:00.102971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001583"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:04.466 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:04.646 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:04.647 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:04.654 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2026-02-21T22:06:04.103410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:04.655 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:04.830 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:04.831 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:04.839 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2026-02-21T22:06:04.103410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:04.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:04.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:04.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:04.841 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:04.841 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:05.056 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:05.056 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":69,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:05.065 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:05.065 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:05.240 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:05.240 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:05.248 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2026-02-21T22:06:04.103410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:08.251 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:08.427 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:08.427 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:08.436 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-21T22:06:08.103965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001502"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:08.437 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:08.614 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:08.615 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:08.623 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-21T22:06:08.103965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001502"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:08.624 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:08.624 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:08.625 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:08.625 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:08.625 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:08.850 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:08.851 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":73,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:08.860 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:08.860 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:09.037 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:09.037 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:09.045 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-21T22:06:08.103965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001502"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:12.047 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:12.226 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:12.226 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:12.235 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2026-02-21T22:06:12.104404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:12.237 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:12.413 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:12.414 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:12.422 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2026-02-21T22:06:12.104404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:12.423 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:12.424 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:12.424 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:12.424 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:12.424 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:12.641 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:12.641 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:12.650 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:12.650 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:12.825 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:12.826 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:12.834 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2026-02-21T22:06:12.104404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:15.836 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:16.012 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:16.013 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:16.021 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2026-02-21T22:06:14.104680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001512"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:16.023 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:16.199 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:16.200 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:16.208 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2026-02-21T22:06:16.104965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001554"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:16.210 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:16.210 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:16.210 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:16.210 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:16.210 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:16.431 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:16.431 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:16.440 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:16.440 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:16.619 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:16.619 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:16.628 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2026-02-21T22:06:16.104965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001554"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:19.630 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:19.807 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:19.807 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:19.816 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2026-02-21T22:06:18.105231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001576"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:19.817 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:19.993 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:19.993 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:20.002 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2026-02-21T22:06:18.105231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001576"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:20.003 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:20.003 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:20.003 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:20.003 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:20.004 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:20.222 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:20.223 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":85,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:20.232 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:20.232 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:20.406 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:20.407 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:20.415 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2026-02-21T22:06:20.105542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001576"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:23.417 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:23.595 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:23.596 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:23.604 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2026-02-21T22:06:22.105799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:23.606 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:23.781 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:23.782 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:23.790 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2026-02-21T22:06:22.105799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:23.791 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:23.791 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:23.791 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:23.792 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:23.792 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:24.010 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:24.010 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":88,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:24.019 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:24.019 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:24.193 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:24.194 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:24.202 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2026-02-21T22:06:24.106070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001665"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:27.205 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:27.381 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:27.381 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:27.389 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2026-02-21T22:06:26.106271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001590"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:27.391 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:27.566 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:27.567 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:27.575 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2026-02-21T22:06:26.106271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001590"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:27.577 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:27.577 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:27.577 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:27.577 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:27.578 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:27.792 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:27.793 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":92,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:27.801 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:27.802 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:27.981 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:27.982 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:27.990 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2026-02-21T22:06:26.106271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001590"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:30.992 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:31.174 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:31.174 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:31.183 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2026-02-21T22:06:30.106828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001597"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:31.184 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:31.358 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:31.358 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:31.366 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2026-02-21T22:06:30.106828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001597"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:31.367 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:31.367 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:31.368 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:31.368 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:31.368 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:31.587 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:31.587 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":96,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:31.596 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:31.597 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:31.773 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:31.773 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:31.781 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2026-02-21T22:06:30.106828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001597"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:34.783 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:34.959 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:34.960 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:34.967 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2026-02-21T22:06:34.107328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:34.969 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:35.146 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:35.146 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:35.154 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2026-02-21T22:06:34.107328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:35.156 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:35.156 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:35.156 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:35.156 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:35.157 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:35.374 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:35.375 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":100,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:35.385 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:35.385 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:35.559 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:35.560 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:35.568 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2026-02-21T22:06:34.107328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:38.570 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:38.746 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:38.746 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:38.754 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2026-02-21T22:06:38.107830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001559"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:38.755 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:38.935 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:38.936 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:38.945 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2026-02-21T22:06:38.107830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001559"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:38.946 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:38.946 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:38.946 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:38.947 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:38.947 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:39.168 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:39.168 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":103,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:39.178 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:39.178 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:39.355 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:39.356 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:39.364 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2026-02-21T22:06:38.107830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001559"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:42.366 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:42.546 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:42.546 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:42.554 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2026-02-21T22:06:42.108351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:42.555 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:42.731 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:42.731 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:42.739 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2026-02-21T22:06:42.108351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:42.740 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:42.740 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:42.740 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:42.740 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:42.740 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:42.957 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:42.957 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":107,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:42.965 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:42.965 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:43.140 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:43.141 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:43.149 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2026-02-21T22:06:42.108351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:46.151 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:46.327 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:46.327 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:46.335 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2026-02-21T22:06:46.108881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001552"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:46.336 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:46.512 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:46.513 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:46.521 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2026-02-21T22:06:46.108881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001552"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:46.523 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:46.523 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:46.523 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:46.523 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:46.524 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:46.744 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:46.744 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":111,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:46.753 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:46.753 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:46.927 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:46.927 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:46.935 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2026-02-21T22:06:46.108881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001552"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:49.936 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:50.112 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:50.112 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:50.121 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2026-02-21T22:06:48.109110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001559"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:50.121 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:50.298 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:50.298 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:50.306 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2026-02-21T22:06:50.109374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001544"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:50.308 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:50.308 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:50.308 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:50.308 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:50.309 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:50.525 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:50.526 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":115,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:50.534 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:50.534 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:50.708 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:50.708 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:50.716 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2026-02-21T22:06:50.109374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001544"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:53.717 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:53.895 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:53.896 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:53.904 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-21T22:06:52.109560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001478"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:53.905 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:54.078 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:54.079 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:54.088 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-21T22:06:52.109560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001478"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:54.088 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:54.088 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:54.088 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:54.088 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:54.088 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:54.308 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:54.309 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":119,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:54.319 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:54.319 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:54.499 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:54.499 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:54.508 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2026-02-21T22:06:54.109796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:57.510 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:57.691 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:57.692 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:57.701 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2026-02-21T22:06:56.110016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:57.701 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:57.881 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:57.881 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:57.890 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2026-02-21T22:06:56.110016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:06:57.891 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:06:57.891 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:06:57.892 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:06:57.892 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:06:57.892 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:06:58.110 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:58.110 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":122,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:06:58.119 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:06:58.119 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:06:58.292 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:06:58.293 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:06:58.301 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2026-02-21T22:06:58.110292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001410"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:01.303 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:01.482 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:01.482 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:01.490 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-21T22:07:00.110506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:01.491 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:01.666 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:01.667 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:01.675 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-21T22:07:00.110506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:01.676 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:01.676 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:01.677 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:01.677 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:01.677 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:01.892 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:01.892 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":126,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:01.900 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:01.901 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:02.076 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:02.076 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:02.084 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-21T22:07:00.110506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:05.086 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:05.264 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:05.264 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:05.273 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2026-02-21T22:07:04.110953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001393"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:05.274 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:05.452 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:05.453 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:05.461 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2026-02-21T22:07:04.110953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001393"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:05.462 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:05.462 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:05.463 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:05.463 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:05.463 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:05.680 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:05.680 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":130,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:05.689 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:05.689 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:05.865 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:05.865 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:05.873 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2026-02-21T22:07:04.110953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001393"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:08.875 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:09.050 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:09.050 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:09.058 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2026-02-21T22:07:08.111422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:09.058 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:09.231 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:09.232 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:09.241 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2026-02-21T22:07:08.111422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:09.243 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:09.243 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:09.243 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:09.243 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:09.243 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:09.460 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:09.460 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":134,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:09.468 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:09.468 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:09.645 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:09.645 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:09.654 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2026-02-21T22:07:08.111422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217221+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217221+0000","last_peered":"2026-02-21T22:05:13.217221+0000","last_clean":"2026-02-21T22:05:13.217221+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217221+0000","last_undegraded":"2026-02-21T22:05:13.217221+0000","last_fullsized":"2026-02-21T22:05:13.217221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220972+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.220972+0000","last_peered":"2026-02-21T22:05:13.220972+0000","last_clean":"2026-02-21T22:05:13.220972+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.220972+0000","last_undegraded":"2026-02-21T22:05:13.220972+0000","last_fullsized":"2026-02-21T22:05:13.220972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221140+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221140+0000","last_peered":"2026-02-21T22:05:13.221140+0000","last_clean":"2026-02-21T22:05:13.221140+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221140+0000","last_undegraded":"2026-02-21T22:05:13.221140+0000","last_fullsized":"2026-02-21T22:05:13.221140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:12.657 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:12.833 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:12.833 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:12.841 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2026-02-21T22:07:12.111992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001486"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:12.842 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:13.016 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:13.017 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:13.024 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2026-02-21T22:07:12.111992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001486"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:13.025 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:13.025 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:13.026 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:13.026 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.220461+0000', 'last_change': '2026-02-21T22:05:13.221382+0000', 'last_active': '2026-02-21T22:05:13.220461+0000', 'last_peered': '2026-02-21T22:05:13.220461+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.220461+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.220461+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:13.026 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:13.241 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:13.242 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:13.251 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:13.252 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:13.431 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:13.431 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:13.440 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2026-02-21T22:07:12.111992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001486"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220534+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220534+0000","last_peered":"2026-02-21T22:05:13.220534+0000","last_clean":"2026-02-21T22:05:13.220534+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220534+0000","last_undegraded":"2026-02-21T22:05:13.220534+0000","last_fullsized":"2026-02-21T22:05:13.220534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.220461+0000","last_change":"2026-02-21T22:05:13.221382+0000","last_active":"2026-02-21T22:05:13.220461+0000","last_peered":"2026-02-21T22:05:13.220461+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.220461+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.220461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220788+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.220788+0000","last_peered":"2026-02-21T22:05:13.220788+0000","last_clean":"2026-02-21T22:05:13.220788+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.220788+0000","last_undegraded":"2026-02-21T22:05:13.220788+0000","last_fullsized":"2026-02-21T22:05:13.220788+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:16.443 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:16.622 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:16.622 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:16.630 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2026-02-21T22:07:16.112625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001672"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:16.631 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:16.806 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:16.807 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:16.815 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2026-02-21T22:07:16.112625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001672"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:16.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:16.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:16.817 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:16.817 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:16.817 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:17.037 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:17.037 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":141,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":1},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:17.046 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:17.047 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:17.223 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:17.223 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:17.232 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2026-02-21T22:07:16.112625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001672"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:20.236 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:20.412 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:20.413 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:20.421 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2026-02-21T22:07:20.113209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001786"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:20.422 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:20.600 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:20.601 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:20.609 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2026-02-21T22:07:20.113209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001786"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:20.611 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:20.611 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-21T22:05:13.221112+0000', 'last_change': '2026-02-21T22:05:13.221475+0000', 'last_active': '2026-02-21T22:05:13.221112+0000', 'last_peered': '2026-02-21T22:05:13.221112+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221112+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221112+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:20.611 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:20.611 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:20.612 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:20.830 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:20.830 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":1},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:20.839 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:20.840 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:21.015 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:21.015 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:21.025 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2026-02-21T22:07:20.113209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001786"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220794+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.220794+0000","last_peered":"2026-02-21T22:05:13.220794+0000","last_clean":"2026-02-21T22:05:13.220794+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.220794+0000","last_undegraded":"2026-02-21T22:05:13.220794+0000","last_fullsized":"2026-02-21T22:05:13.220794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":15,"reported_epoch":17,"state":"active+recovering+degraded","last_fresh":"2026-02-21T22:05:13.221112+0000","last_change":"2026-02-21T22:05:13.221475+0000","last_active":"2026-02-21T22:05:13.221112+0000","last_peered":"2026-02-21T22:05:13.221112+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221112+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:24.027 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:24.205 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:24.206 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:24.216 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2026-02-21T22:07:24.113734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001741"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:24.217 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:24.393 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:24.394 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:24.402 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2026-02-21T22:07:24.113734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001741"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:24.404 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:24.404 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:24.404 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:24.404 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:24.405 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:24.623 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:24.623 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":149,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:24.632 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:24.632 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:24.808 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:24.809 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:24.817 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2026-02-21T22:07:24.113734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001741"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:27.819 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:28.000 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:28.001 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:28.009 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2026-02-21T22:07:26.113949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001643"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:28.009 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:28.184 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:28.185 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:28.193 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2026-02-21T22:07:28.114257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001631"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:28.195 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:28.195 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:28.195 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:28.195 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:28.196 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:28.409 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:28.410 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":153,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:28.419 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:28.419 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:28.597 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:28.598 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:28.605 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2026-02-21T22:07:28.114257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001631"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:31.607 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:31.785 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:31.785 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:31.794 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2026-02-21T22:07:30.114492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001579"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:31.795 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:31.969 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:31.970 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:31.977 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2026-02-21T22:07:30.114492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001579"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:31.979 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:31.979 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:31.979 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:31.979 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:31.980 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:32.197 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:32.197 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":157,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:32.206 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:32.207 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:32.385 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:32.386 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:32.394 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2026-02-21T22:07:32.114759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001550"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:35.397 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:35.574 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:35.575 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:35.582 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2026-02-21T22:07:34.115003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001565"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:35.584 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:35.761 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:35.761 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:35.771 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2026-02-21T22:07:34.115003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001565"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:35.772 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:35.772 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:35.772 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:35.772 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:35.773 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:35.990 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:35.991 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":160,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:36.000 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:36.000 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:36.179 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:36.179 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:36.189 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2026-02-21T22:07:36.115265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001530"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:39.191 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:39.367 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:39.367 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:39.374 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2026-02-21T22:07:38.115487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001537"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:39.375 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:39.551 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:39.552 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:39.560 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2026-02-21T22:07:38.115487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001537"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:39.561 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:39.561 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:39.561 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:39.561 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:39.561 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:39.778 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:39.778 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":164,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:39.787 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:39.787 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:39.962 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:39.963 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:39.971 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2026-02-21T22:07:38.115487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001537"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:42.973 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:43.153 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:43.153 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:43.162 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T22:07:42.115989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:43.163 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:43.342 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:43.342 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:43.351 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T22:07:42.115989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:43.352 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:43.352 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:43.353 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:43.353 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:43.353 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:43.569 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:43.569 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":168,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:43.578 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:43.578 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:43.754 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:43.755 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:43.763 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T22:07:42.115989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:46.765 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:46.942 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:46.942 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:46.950 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2026-02-21T22:07:46.116455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:46.950 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:47.124 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:47.124 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:47.133 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2026-02-21T22:07:46.116455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:47.134 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:47.134 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:47.134 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:47.134 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:47.134 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:47.350 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:47.351 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":172,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:47.359 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:47.360 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:47.536 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:47.537 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:47.545 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2026-02-21T22:07:46.116455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:50.548 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:50.727 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:50.727 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:50.735 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2026-02-21T22:07:50.116947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001459"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:50.737 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:50.912 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:50.913 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:50.921 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2026-02-21T22:07:50.116947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001459"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:50.924 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:50.924 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:50.924 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:50.925 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:50.925 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:51.143 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:51.143 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":175,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:51.152 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:51.152 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:51.330 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:51.330 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:51.339 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2026-02-21T22:07:50.116947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001459"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:54.341 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:54.518 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:54.518 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:54.527 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2026-02-21T22:07:54.117473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:54.527 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:54.704 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:54.704 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:54.712 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2026-02-21T22:07:54.117473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:54.713 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:54.713 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:54.714 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:54.714 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:54.714 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:54.932 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:54.932 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":179,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:54.941 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:54.942 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:55.117 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:55.117 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:55.126 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2026-02-21T22:07:54.117473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:58.128 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:58.307 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:58.308 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:58.316 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2026-02-21T22:07:58.117869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:58.317 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:58.492 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:58.492 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:58.500 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2026-02-21T22:07:58.117869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:07:58.501 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:07:58.501 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:07:58.501 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:07:58.501 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:07:58.501 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:07:58.717 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:58.717 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":183,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:07:58.726 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:07:58.726 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:07:58.904 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:07:58.904 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:07:58.912 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2026-02-21T22:07:58.117869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:01.915 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:02.094 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:02.094 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:02.102 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2026-02-21T22:08:00.118086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:02.103 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:02.275 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:02.276 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:02.284 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2026-02-21T22:08:02.118347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:02.286 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:02.286 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:02.286 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:02.286 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:02.287 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:02.503 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:02.503 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":187,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:02.512 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:02.512 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:02.684 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:02.685 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:02.693 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2026-02-21T22:08:02.118347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:05.695 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:05.872 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:05.872 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:05.880 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2026-02-21T22:08:04.118594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:05.881 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:06.054 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:06.054 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:06.063 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2026-02-21T22:08:04.118594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:06.063 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:06.063 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:06.063 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:06.063 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:06.064 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:06.280 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:06.280 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":191,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:06.289 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:06.289 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:06.464 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:06.465 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:06.473 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2026-02-21T22:08:06.118862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:09.476 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:09.656 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:09.656 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:09.665 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2026-02-21T22:08:08.119095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001438"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:09.666 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:09.841 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:09.841 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:09.849 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2026-02-21T22:08:08.119095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001438"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:09.850 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:09.850 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:09.850 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:09.851 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:09.851 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:10.068 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:10.069 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":194,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:10.078 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:10.078 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:10.254 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:10.254 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:10.263 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2026-02-21T22:08:10.119348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001479"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:13.265 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:13.441 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:13.442 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:13.450 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2026-02-21T22:08:12.119571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001485"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:13.452 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:13.628 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:13.629 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:13.637 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2026-02-21T22:08:12.119571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001485"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:13.639 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:13.639 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:13.639 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:13.639 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:13.640 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:13.856 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:13.856 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":198,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:13.864 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:13.865 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:14.043 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:14.044 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:14.052 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2026-02-21T22:08:12.119571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001485"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:17.054 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:17.231 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:17.232 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:17.240 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-21T22:08:16.120063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:17.242 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:17.417 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:17.417 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:17.425 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-21T22:08:16.120063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:17.426 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:17.427 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:17.427 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:17.427 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:17.427 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:17.644 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:17.645 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":202,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:17.653 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:17.653 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:17.828 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:17.828 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:17.836 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-21T22:08:16.120063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:20.838 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:21.015 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:21.015 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:21.023 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2026-02-21T22:08:20.120538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001442"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:21.024 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:21.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:21.198 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:21.206 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2026-02-21T22:08:20.120538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001442"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:21.207 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:21.208 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:21.208 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:21.208 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:21.208 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:21.427 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:21.427 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":206,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:21.435 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:21.436 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:21.613 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:21.614 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:21.622 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2026-02-21T22:08:20.120538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001442"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:24.623 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:24.801 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:24.801 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:24.809 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-21T22:08:24.121053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:24.810 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:24.985 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:24.985 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:24.994 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-21T22:08:24.121053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:24.994 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:24.994 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:24.994 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:24.994 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:24.994 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:25.209 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:25.209 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":210,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:25.217 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:25.217 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:25.392 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:25.392 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:25.400 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-21T22:08:24.121053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:28.401 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:28.579 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:28.579 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:28.587 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2026-02-21T22:08:28.121562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001498"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:28.588 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:28.762 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:28.762 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:28.771 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2026-02-21T22:08:28.121562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001498"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:28.771 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:28.771 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:28.771 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:28.772 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:28.772 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:28.988 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:28.988 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":213,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:28.997 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:28.997 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:29.173 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:29.173 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:29.181 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2026-02-21T22:08:28.121562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001498"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:32.183 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:32.361 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:32.362 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:32.371 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2026-02-21T22:08:32.122062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:32.371 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:32.547 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:32.547 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:32.555 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2026-02-21T22:08:32.122062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:32.555 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:32.555 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:32.556 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:32.556 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:32.556 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:32.771 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:32.771 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":217,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:32.780 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:32.780 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:32.954 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:32.955 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:32.962 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2026-02-21T22:08:32.122062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:35.964 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:36.145 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:36.146 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:36.154 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2026-02-21T22:08:36.122528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:36.155 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:36.336 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:36.336 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:36.345 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2026-02-21T22:08:36.122528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:36.346 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:36.347 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:36.347 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:36.347 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:36.347 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:36.565 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:36.565 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:36.574 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:36.574 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:36.752 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:36.753 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:36.762 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2026-02-21T22:08:36.122528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:39.764 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:39.942 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:39.942 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:39.951 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2026-02-21T22:08:38.122743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001476"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:39.952 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:40.128 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:40.128 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:40.137 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2026-02-21T22:08:40.122974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:40.138 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:40.138 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:40.138 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:40.138 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:40.138 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:40.354 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:40.354 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":225,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:40.363 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:40.363 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:40.537 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:40.537 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:40.545 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2026-02-21T22:08:40.122974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:43.546 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:43.727 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:43.728 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:43.737 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2026-02-21T22:08:42.123203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:43.738 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:43.912 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:43.912 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:43.920 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2026-02-21T22:08:42.123203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:43.921 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:43.921 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:43.921 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:43.921 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:43.921 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:44.132 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:44.132 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:44.141 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:44.141 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:44.315 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:44.316 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:44.325 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2026-02-21T22:08:44.123482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:47.327 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:47.504 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:47.504 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:47.512 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2026-02-21T22:08:46.123707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:47.513 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:47.687 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:47.687 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:47.695 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2026-02-21T22:08:46.123707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:47.695 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:47.695 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:47.695 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:47.695 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:47.696 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:47.912 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:47.912 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":232,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:47.923 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:47.923 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:48.099 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:48.099 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:48.107 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2026-02-21T22:08:46.123707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:51.108 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:51.288 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:51.288 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:51.297 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-21T22:08:50.124130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:51.299 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:51.474 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:51.475 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:51.483 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-21T22:08:50.124130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:51.485 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:51.485 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:51.485 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:51.485 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:51.486 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:51.705 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:51.705 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":236,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:51.715 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:51.716 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:51.893 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:51.894 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:51.902 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-21T22:08:50.124130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:54.904 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:55.081 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:55.082 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:55.091 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2026-02-21T22:08:54.124526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001322"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:55.092 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:55.267 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:55.268 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:55.276 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2026-02-21T22:08:54.124526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001322"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:55.278 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:55.278 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:55.278 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:55.278 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:55.279 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:55.493 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:55.493 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":240,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:55.501 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:55.502 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:55.676 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:55.677 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:55.685 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2026-02-21T22:08:54.124526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001322"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:58.686 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:58.862 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:58.862 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:58.871 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-21T22:08:58.124905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001197"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:58.871 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:59.044 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:59.044 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:59.052 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-21T22:08:58.124905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001197"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:08:59.053 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:08:59.053 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:08:59.053 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:08:59.053 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:08:59.053 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:08:59.271 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:59.271 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":244,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:08:59.281 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:08:59.281 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:08:59.454 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:08:59.454 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:08:59.462 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-21T22:08:58.124905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001197"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:02.463 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:02.638 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:02.638 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:02.646 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2026-02-21T22:09:02.125236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001105"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:02.647 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:02.823 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:02.823 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:02.831 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2026-02-21T22:09:02.125236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001105"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:02.832 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:02.832 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:02.832 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:02.832 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:02.833 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:03.050 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:03.050 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":247,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:03.059 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:03.059 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:03.237 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:03.237 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:03.246 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2026-02-21T22:09:02.125236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001105"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:06.248 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:06.425 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:06.426 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:06.433 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2026-02-21T22:09:06.125649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001123"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:06.435 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:06.610 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:06.610 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:06.618 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2026-02-21T22:09:06.125649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001123"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:06.620 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:06.620 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:06.620 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:06.620 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:06.621 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:06.841 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:06.841 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":251,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:06.851 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:06.851 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:07.026 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:07.026 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:07.035 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2026-02-21T22:09:06.125649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001123"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:10.038 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:10.216 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:10.216 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:10.225 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2026-02-21T22:09:10.126090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001185"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:10.226 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:10.403 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:10.404 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:10.412 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2026-02-21T22:09:10.126090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001185"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:10.414 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:10.414 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:10.414 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:10.414 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:10.415 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:10.635 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:10.635 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":255,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:10.645 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:10.645 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:10.823 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:10.824 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:10.831 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2026-02-21T22:09:10.126090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001185"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:13.834 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:14.010 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:14.010 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:14.018 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2026-02-21T22:09:12.126293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001217"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:14.019 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:14.192 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:14.192 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:14.201 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2026-02-21T22:09:14.126558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:14.202 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:14.202 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:14.203 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:14.203 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:14.203 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:14.421 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:14.421 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":259,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:14.431 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:14.431 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:14.607 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:14.607 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:14.615 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2026-02-21T22:09:14.126558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.217250+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:05:13.217250+0000","last_peered":"2026-02-21T22:05:13.217250+0000","last_clean":"2026-02-21T22:05:13.217250+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:05:13.217250+0000","last_undegraded":"2026-02-21T22:05:13.217250+0000","last_fullsized":"2026-02-21T22:05:13.217250+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221346+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:05:13.221346+0000","last_peered":"2026-02-21T22:05:13.221346+0000","last_clean":"2026-02-21T22:05:13.221346+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:05:13.221346+0000","last_undegraded":"2026-02-21T22:05:13.221346+0000","last_fullsized":"2026-02-21T22:05:13.221346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221427+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:05:13.221427+0000","last_peered":"2026-02-21T22:05:13.221427+0000","last_clean":"2026-02-21T22:05:13.221427+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:05:13.221427+0000","last_undegraded":"2026-02-21T22:05:13.221427+0000","last_fullsized":"2026-02-21T22:05:13.221427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221117+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:05:13.221117+0000","last_peered":"2026-02-21T22:05:13.221117+0000","last_clean":"2026-02-21T22:05:13.221117+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:05:13.221117+0000","last_undegraded":"2026-02-21T22:05:13.221117+0000","last_fullsized":"2026-02-21T22:05:13.221117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:17.617 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:17.796 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:17.797 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:17.805 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2026-02-21T22:09:16.126807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001373"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:17.806 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:17.982 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:17.982 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:17.991 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2026-02-21T22:09:16.126807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001373"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:17.991 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:17.991 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:17.991 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:17.991 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:17.991 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:18.207 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:18.207 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":263,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:18.216 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:18.216 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:18.392 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:18.392 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:18.400 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2026-02-21T22:09:18.127064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:21.402 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:21.579 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:21.580 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:21.588 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2026-02-21T22:09:20.127271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001404"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:21.589 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:21.765 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:21.765 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:21.773 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2026-02-21T22:09:20.127271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001404"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":11,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221382+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:05:13.221382+0000","last_peered":"2026-02-21T22:05:13.221382+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:05:13.221382+0000","last_undegraded":"2026-02-21T22:05:13.217264+0000","last_fullsized":"2026-02-21T22:05:13.221382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:21.775 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:21.775 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:21.775 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:21.775 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 11, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221382+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:05:13.221382+0000', 'last_peered': '2026-02-21T22:05:13.221382+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:05:13.221382+0000', 'last_undegraded': '2026-02-21T22:05:13.217264+0000', 'last_fullsized': '2026-02-21T22:05:13.221382+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:21.776 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:21.992 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:21.993 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":266,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:22.001 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:22.002 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:22.178 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:22.179 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:22.187 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2026-02-21T22:09:22.127522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:25.190 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:25.369 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:25.369 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:25.377 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2026-02-21T22:09:24.127740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001447"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:25.379 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:25.553 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:25.553 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:25.560 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2026-02-21T22:09:24.127740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001447"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:25.561 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:25.561 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:05:13.221475+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:05:13.221475+0000', 'last_peered': '2026-02-21T22:05:13.221475+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:05:13.221475+0000', 'last_undegraded': '2026-02-21T22:05:13.217128+0000', 'last_fullsized': '2026-02-21T22:05:13.221475+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:25.561 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:25.561 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:25.561 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:25.781 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:25.781 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":270,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:25.790 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:25.790 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:25.966 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:25.966 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:25.974 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2026-02-21T22:09:24.127740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001447"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.220963+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:05:13.220963+0000","last_peered":"2026-02-21T22:05:13.220963+0000","last_clean":"2026-02-21T22:05:13.220963+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:05:13.220963+0000","last_undegraded":"2026-02-21T22:05:13.220963+0000","last_fullsized":"2026-02-21T22:05:13.220963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:05:13.221475+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:05:13.221475+0000","last_peered":"2026-02-21T22:05:13.221475+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:05:13.221475+0000","last_undegraded":"2026-02-21T22:05:13.217128+0000","last_fullsized":"2026-02-21T22:05:13.221475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:28.975 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:29.153 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:29.153 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:29.161 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2026-02-21T22:09:28.128185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:29.163 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:29.339 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:29.339 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:29.348 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2026-02-21T22:09:28.128185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:29.349 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:29.349 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:29.350 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:29.350 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:29.350 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:29.571 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:29.571 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":274,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:29.581 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:29.581 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:29.759 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:29.760 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:29.768 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2026-02-21T22:09:28.128185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:05:13.221236+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:05:13.221236+0000","last_peered":"2026-02-21T22:05:13.221236+0000","last_clean":"2026-02-21T22:05:13.221236+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:05:13.221236+0000","last_undegraded":"2026-02-21T22:05:13.221236+0000","last_fullsized":"2026-02-21T22:05:13.221236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:32.771 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:32.952 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:32.953 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:32.961 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-21T22:09:32.128649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:32.962 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:33.140 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:33.141 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:33.149 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-21T22:09:32.128649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:33.150 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:33.151 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:33.151 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:33.151 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:33.151 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:33.371 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:33.372 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:33.382 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:33.382 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:33.562 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:33.562 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:33.570 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-21T22:09:32.128649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:36.573 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:36.750 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:36.751 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:36.759 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-21T22:09:36.129121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:36.760 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:36.935 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:36.936 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:36.944 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-21T22:09:36.129121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:36.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:36.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:36.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:36.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:36.946 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:37.167 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:37.167 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":281,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:37.176 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:37.176 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:37.353 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:37.353 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:37.362 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-21T22:09:36.129121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:40.363 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:40.539 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:40.539 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:40.548 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2026-02-21T22:09:40.129629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:40.548 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:40.724 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:40.724 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:40.732 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2026-02-21T22:09:40.129629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:40.734 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:40.734 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:40.734 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:40.734 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:40.734 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:40.952 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:40.952 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":285,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:40.962 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:40.962 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:41.137 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:41.137 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:41.146 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2026-02-21T22:09:40.129629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:44.148 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:44.325 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:44.325 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:44.333 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2026-02-21T22:09:44.130079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:44.335 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:44.511 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:44.512 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:44.520 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2026-02-21T22:09:44.130079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:44.522 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:44.522 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:44.522 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:44.522 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:44.522 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:44.740 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:44.741 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":289,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:44.749 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:44.749 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:44.927 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:44.930 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:44.936 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2026-02-21T22:09:44.130079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:47.938 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:48.114 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:48.115 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:48.122 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2026-02-21T22:09:46.130303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:48.124 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:48.299 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:48.300 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:48.309 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2026-02-21T22:09:48.130567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:48.310 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:48.310 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:48.310 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:48.311 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:48.311 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:48.528 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:48.528 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":293,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:48.537 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:48.537 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:48.715 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:48.716 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:48.724 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2026-02-21T22:09:48.130567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:51.726 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:51.905 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:51.905 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:51.913 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2026-02-21T22:09:50.130741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:51.915 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:52.089 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:52.090 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:52.097 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2026-02-21T22:09:50.130741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:52.099 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:52.099 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:52.099 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:52.099 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:52.100 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:52.318 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:52.318 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":297,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:52.328 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:52.328 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:52.504 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:52.504 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:52.512 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2026-02-21T22:09:52.130913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001283"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:55.514 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:55.695 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:55.695 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:55.703 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2026-02-21T22:09:54.131124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001310"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:55.705 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:55.879 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:55.880 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:55.887 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2026-02-21T22:09:54.131124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001310"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:55.889 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:55.889 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:55.889 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:55.889 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:55.889 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:56.106 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:56.106 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":300,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:56.115 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:56.115 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:56.292 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:56.293 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:56.301 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2026-02-21T22:09:56.131367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001287"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:59.304 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:59.480 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:59.481 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:59.489 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2026-02-21T22:09:58.131562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001259"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:59.490 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:09:59.666 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:59.667 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:09:59.675 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2026-02-21T22:09:58.131562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001259"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:09:59.677 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:09:59.677 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:09:59.677 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:09:59.677 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:09:59.677 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:09:59.898 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:09:59.898 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":304,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:09:59.907 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:09:59.907 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:00.081 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:00.081 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:00.089 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2026-02-21T22:09:58.131562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001259"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:03.091 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:03.268 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:03.269 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:03.277 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2026-02-21T22:10:02.131958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001216"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:03.279 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:03.455 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:03.456 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:03.464 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2026-02-21T22:10:02.131958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001216"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:03.466 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:03.466 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:03.466 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:03.466 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:03.467 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:03.681 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:03.681 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":308,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:03.690 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:03.690 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:03.869 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:03.869 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:03.877 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2026-02-21T22:10:02.131958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001216"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:06.879 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:07.059 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:07.059 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:07.068 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2026-02-21T22:10:06.132418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:07.068 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:07.245 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:07.245 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:07.254 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2026-02-21T22:10:06.132418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:07.255 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:07.255 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:07.255 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:07.255 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:07.255 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:07.472 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:07.472 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":312,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:07.481 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:07.482 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:07.657 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:07.657 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:07.665 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2026-02-21T22:10:06.132418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:10.667 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:10.844 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:10.845 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:10.853 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2026-02-21T22:10:10.132905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001342"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:10.855 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:11.032 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:11.032 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:11.040 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2026-02-21T22:10:10.132905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001342"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:11.042 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:11.042 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:11.042 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:11.042 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:11.043 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:11.261 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:11.261 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:11.270 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:11.271 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:11.448 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:11.448 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:11.457 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2026-02-21T22:10:10.132905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001342"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:14.459 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:14.635 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:14.635 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:14.643 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2026-02-21T22:10:14.133277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001319"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:14.645 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:14.818 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:14.818 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:14.826 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2026-02-21T22:10:14.133277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001319"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:14.827 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:14.827 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:14.827 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:14.827 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:14.828 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:15.047 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:15.048 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":319,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:15.058 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:15.058 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:15.230 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:15.231 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:15.239 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2026-02-21T22:10:14.133277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001319"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:18.242 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:18.421 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:18.422 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:18.430 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2026-02-21T22:10:18.133787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:18.431 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:18.608 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:18.609 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:18.617 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2026-02-21T22:10:18.133787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:18.618 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:18.618 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:18.618 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:18.618 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:18.619 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:18.836 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:18.836 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":323,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:18.845 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:18.845 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:19.019 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:19.020 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:19.028 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2026-02-21T22:10:18.133787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:22.029 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:22.208 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:22.208 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:22.217 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2026-02-21T22:10:22.134288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001382"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:22.218 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:22.393 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:22.393 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:22.401 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2026-02-21T22:10:22.134288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001382"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:22.401 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:22.402 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:22.402 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:22.402 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:22.402 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:22.618 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:22.618 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":327,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:22.628 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:22.628 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:22.804 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:22.804 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:22.812 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2026-02-21T22:10:22.134288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001382"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:25.814 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:25.989 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:25.989 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:25.996 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2026-02-21T22:10:24.134528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:25.997 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:26.172 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:26.172 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:26.181 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2026-02-21T22:10:26.134782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001504"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:26.183 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:26.183 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:26.183 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:26.184 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:26.184 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:26.403 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:26.404 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":331,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:26.413 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:26.413 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:26.587 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:26.587 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:26.595 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2026-02-21T22:10:26.134782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001504"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:29.596 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:29.771 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:29.771 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:29.779 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2026-02-21T22:10:28.135008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001508"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:29.781 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:29.955 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:29.955 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:29.963 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2026-02-21T22:10:28.135008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001508"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:29.963 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:29.963 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:29.963 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:29.963 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:29.963 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:30.185 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:30.185 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":335,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:30.194 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:30.194 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:30.369 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:30.369 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:30.378 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2026-02-21T22:10:30.135248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001461"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:33.380 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:33.562 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:33.562 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:33.571 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2026-02-21T22:10:32.135461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:33.572 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:33.748 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:33.749 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:33.757 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2026-02-21T22:10:32.135461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:33.757 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:33.757 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:33.757 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:33.757 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:33.757 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:33.973 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:33.973 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":338,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:33.981 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:33.981 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:34.159 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:34.159 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:34.168 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2026-02-21T22:10:34.135703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:37.170 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:37.346 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:37.347 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:37.355 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-21T22:10:36.135896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:37.355 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:37.533 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:37.534 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:37.543 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-21T22:10:36.135896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:37.544 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:37.544 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:37.544 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:37.544 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:37.545 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:37.763 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:37.764 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":342,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:37.773 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:37.773 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:37.947 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:37.947 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:37.955 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-21T22:10:36.135896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:40.957 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:41.130 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:41.131 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:41.139 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2026-02-21T22:10:40.136346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001337"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:41.140 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:41.318 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:41.319 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:41.327 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2026-02-21T22:10:40.136346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001337"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:41.329 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:41.329 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:41.329 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:41.329 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:41.330 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:41.546 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:41.547 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":346,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:41.555 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:41.555 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:41.733 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:41.733 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:41.742 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2026-02-21T22:10:40.136346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001337"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:44.743 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:44.924 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:44.925 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:44.933 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-21T22:10:44.136712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001251"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:44.934 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:45.112 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:45.113 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:45.121 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-21T22:10:44.136712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001251"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:45.121 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:45.121 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:45.121 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:45.121 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:45.122 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:45.335 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:45.335 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":350,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:45.345 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:45.345 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:45.520 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:45.520 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:45.528 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-21T22:10:44.136712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001251"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:48.531 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:48.711 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:48.712 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:48.721 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2026-02-21T22:10:48.137190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:48.722 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:48.898 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:48.899 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:48.906 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2026-02-21T22:10:48.137190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:48.909 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:48.909 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:48.910 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:48.910 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:48.910 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:49.127 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:49.128 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":353,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:49.137 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:49.137 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:49.317 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:49.318 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:49.327 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2026-02-21T22:10:48.137190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:52.330 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:52.508 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:52.509 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:52.517 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-21T22:10:52.137696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:52.518 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:52.697 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:52.698 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:52.706 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-21T22:10:52.137696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:52.708 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:52.708 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:52.708 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:52.708 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:52.709 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:52.928 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:52.928 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":357,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:52.938 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:52.938 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:53.117 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:53.118 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:53.126 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-21T22:10:52.137696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:56.129 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:56.308 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:56.309 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:56.318 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2026-02-21T22:10:56.138201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:56.319 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:56.496 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:56.496 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:56.504 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2026-02-21T22:10:56.138201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:56.505 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:10:56.505 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:10:56.505 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:10:56.505 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:10:56.505 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:10:56.725 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:56.725 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":361,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:10:56.734 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:10:56.735 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:10:56.910 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:10:56.910 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:10:56.919 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2026-02-21T22:10:56.138201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:10:59.921 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:00.098 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:00.098 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:00.107 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2026-02-21T22:10:58.138374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001452"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:00.107 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:00.286 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:00.287 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:00.295 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2026-02-21T22:11:00.138558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:00.297 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:00.297 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:00.297 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:00.297 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:00.298 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:00.513 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:00.514 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":365,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:00.522 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:00.522 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:00.698 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:00.699 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:00.706 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2026-02-21T22:11:00.138558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:03.708 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:03.887 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:03.887 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:03.896 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2026-02-21T22:11:02.138796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:03.897 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:04.075 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:04.076 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:04.084 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2026-02-21T22:11:02.138796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:04.086 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:04.086 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:04.086 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:04.086 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:04.087 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:04.304 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:04.304 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":369,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:04.312 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:04.312 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:04.488 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:04.489 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:04.496 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2026-02-21T22:11:04.139075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001379"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:07.498 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:07.679 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:07.680 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:07.688 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2026-02-21T22:11:06.139308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001354"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:07.690 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:07.867 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:07.868 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:07.876 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2026-02-21T22:11:06.139308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001354"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:07.877 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:07.877 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:07.878 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:07.878 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:07.878 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:08.094 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:08.094 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":372,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:08.102 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:08.102 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:08.280 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:08.281 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:08.289 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2026-02-21T22:11:08.139570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:11.290 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:11.467 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:11.468 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:11.475 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-21T22:11:10.139750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001376"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:11.477 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:11.654 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:11.655 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:11.663 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-21T22:11:10.139750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001376"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:11.664 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:11.664 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:11.665 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:11.665 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:11.665 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:11.882 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:11.882 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":376,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:11.891 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:11.891 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:12.066 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:12.067 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:12.075 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-21T22:11:10.139750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001376"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:15.077 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:15.252 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:15.252 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:15.261 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-21T22:11:14.140143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001347"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:15.262 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:15.440 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:15.440 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:15.448 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-21T22:11:14.140143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001347"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:15.449 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:15.450 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:15.450 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:15.450 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:15.450 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:15.665 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:15.665 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":380,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:15.674 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:15.674 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:15.846 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:15.847 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:15.855 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-21T22:11:14.140143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001347"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200127+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:07:15.200127+0000","last_peered":"2026-02-21T22:07:15.200127+0000","last_clean":"2026-02-21T22:07:15.200127+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:07:15.200127+0000","last_undegraded":"2026-02-21T22:07:15.200127+0000","last_fullsized":"2026-02-21T22:07:15.200127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:18.857 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:19.032 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:19.033 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:19.041 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2026-02-21T22:11:18.140518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001210"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:19.042 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:19.216 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:19.217 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:19.225 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2026-02-21T22:11:18.140518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001210"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:19.226 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:19.226 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:19.227 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:19.227 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:19.227 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:19.443 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:19.443 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":384,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:19.453 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:19.453 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:19.630 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:19.631 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:19.639 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2026-02-21T22:11:18.140518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001210"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.199627+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:07:10.199627+0000","last_peered":"2026-02-21T22:07:10.199627+0000","last_clean":"2026-02-21T22:07:10.199627+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:07:10.199627+0000","last_undegraded":"2026-02-21T22:07:10.199627+0000","last_fullsized":"2026-02-21T22:07:10.199627+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200086+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:07:10.200086+0000","last_peered":"2026-02-21T22:07:10.200086+0000","last_clean":"2026-02-21T22:07:10.200086+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:07:10.200086+0000","last_undegraded":"2026-02-21T22:07:10.200086+0000","last_fullsized":"2026-02-21T22:07:10.200086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:10.200094+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:07:10.200094+0000","last_peered":"2026-02-21T22:07:10.200094+0000","last_clean":"2026-02-21T22:07:10.200094+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:07:10.200094+0000","last_undegraded":"2026-02-21T22:07:10.200094+0000","last_fullsized":"2026-02-21T22:07:10.200094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:22.642 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:22.822 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:22.822 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:22.830 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2026-02-21T22:11:22.141004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:22.832 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:23.007 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:23.007 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:23.015 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2026-02-21T22:11:22.141004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:23.017 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:23.017 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:23.017 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:23.017 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:15.200569+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:07:15.200569+0000', 'last_peered': '2026-02-21T22:07:15.200569+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:07:15.200569+0000', 'last_undegraded': '2026-02-21T22:07:15.200569+0000', 'last_fullsized': '2026-02-21T22:07:15.200569+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:23.018 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:23.233 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:23.234 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":388,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:23.243 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:23.243 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:23.420 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:23.420 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:23.428 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2026-02-21T22:11:22.141004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:15.200569+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:07:15.200569+0000","last_peered":"2026-02-21T22:07:15.200569+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:07:15.200569+0000","last_undegraded":"2026-02-21T22:07:15.200569+0000","last_fullsized":"2026-02-21T22:07:15.200569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:26.430 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:26.609 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:26.610 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:26.619 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2026-02-21T22:11:26.141398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:26.620 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:26.797 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:26.797 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:26.805 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2026-02-21T22:11:26.141398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:26.806 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:26.806 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:26.806 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:26.807 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:26.807 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:27.025 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:27.025 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":391,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:27.034 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:27.035 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:27.211 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:27.212 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:27.220 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2026-02-21T22:11:26.141398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001254"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:30.223 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:30.400 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:30.401 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:30.408 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-21T22:11:30.141922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001404"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:30.410 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:30.585 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:30.586 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:30.594 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-21T22:11:30.141922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001404"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:30.595 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:30.595 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:07:20.200622+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:07:20.200622+0000', 'last_peered': '2026-02-21T22:07:20.200622+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:07:20.200622+0000', 'last_undegraded': '2026-02-21T22:07:20.200622+0000', 'last_fullsized': '2026-02-21T22:07:20.200622+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:30.596 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:30.596 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:30.596 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:30.819 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:30.820 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":395,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:30.828 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:30.829 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:31.004 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:31.005 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:31.013 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-21T22:11:30.141922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001404"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:07:20.200622+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:07:20.200622+0000","last_peered":"2026-02-21T22:07:20.200622+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:07:20.200622+0000","last_undegraded":"2026-02-21T22:07:20.200622+0000","last_fullsized":"2026-02-21T22:07:20.200622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:34.015 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:34.193 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:34.193 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:34.203 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-21T22:11:34.142397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:34.204 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:34.382 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:34.382 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:34.390 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-21T22:11:34.142397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:34.392 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:34.392 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:34.392 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:34.392 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:34.393 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:34.611 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:34.612 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":399,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:34.621 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:34.621 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:34.796 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:34.796 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:34.804 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-21T22:11:34.142397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:15.200640+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:07:15.200640+0000","last_peered":"2026-02-21T22:07:15.200640+0000","last_clean":"2026-02-21T22:07:15.200640+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:07:15.200640+0000","last_undegraded":"2026-02-21T22:07:15.200640+0000","last_fullsized":"2026-02-21T22:07:15.200640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:37.806 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:37.983 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:37.983 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:37.992 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2026-02-21T22:11:36.142620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:37.993 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:38.166 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:38.166 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:38.174 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2026-02-21T22:11:38.142900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001502"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:38.175 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:38.175 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:38.175 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:38.175 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:38.176 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:38.394 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:38.395 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:38.403 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:38.403 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:38.580 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:38.580 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:38.589 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2026-02-21T22:11:38.142900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001502"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:41.591 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:41.768 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:41.769 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:41.777 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2026-02-21T22:11:40.143137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001478"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:41.778 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:41.951 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:41.951 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:41.959 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2026-02-21T22:11:40.143137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001478"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:07:20.201162+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:07:20.201162+0000","last_peered":"2026-02-21T22:07:20.201162+0000","last_clean":"2026-02-21T22:07:20.201162+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:07:20.201162+0000","last_undegraded":"2026-02-21T22:07:20.201162+0000","last_fullsized":"2026-02-21T22:07:20.201162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:41.961 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:41.961 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:41.961 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:41.961 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:41.962 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:42.180 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:42.181 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":407,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:42.190 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:42.190 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:42.369 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:42.369 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:42.378 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2026-02-21T22:11:42.143410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:45.380 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:45.559 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:45.560 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:45.567 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2026-02-21T22:11:44.143651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001534"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:45.569 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:45.743 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:45.744 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:45.752 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2026-02-21T22:11:44.143651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001534"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:45.753 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:45.753 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:45.754 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:45.754 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:45.754 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:45.973 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:45.974 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:45.983 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:45.983 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:46.160 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:46.160 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:46.169 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2026-02-21T22:11:46.143914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001516"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:49.170 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:49.348 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:49.348 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:49.356 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-21T22:11:48.144143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001522"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:49.358 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:49.538 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:49.539 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:49.547 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-21T22:11:48.144143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001522"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:49.550 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:49.550 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:49.551 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:49.551 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:49.551 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:49.769 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:49.770 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":414,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:49.778 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:49.779 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:49.954 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:49.955 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:49.963 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-21T22:11:48.144143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001522"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:52.965 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:53.145 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:53.145 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:53.153 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-21T22:11:52.144699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001562"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:53.155 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:53.331 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:53.331 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:53.339 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-21T22:11:52.144699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001562"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:53.340 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:53.340 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:53.340 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:53.340 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:53.340 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:53.562 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:53.562 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":418,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:53.573 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:53.573 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:53.750 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:53.751 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:53.759 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-21T22:11:52.144699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001562"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:56.761 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:56.938 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:56.938 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:56.946 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-21T22:11:56.145252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001601"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:56.947 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:57.125 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:57.126 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:57.134 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-21T22:11:56.145252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001601"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:11:57.136 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:11:57.136 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:11:57.136 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:11:57.136 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:11:57.137 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:11:57.355 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:57.356 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":422,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:11:57.366 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:11:57.366 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:11:57.543 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:11:57.544 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:11:57.552 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-21T22:11:56.145252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001601"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:00.555 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:00.732 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:00.732 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:00.740 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-21T22:12:00.145747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001603"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:00.741 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:00.914 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:00.915 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:00.923 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-21T22:12:00.145747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001603"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:00.924 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:00.924 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:00.924 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:00.924 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:00.924 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:01.140 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:01.141 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":425,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:01.150 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:01.150 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:01.325 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:01.325 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:01.334 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-21T22:12:00.145747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001603"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:04.335 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:04.515 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:04.515 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:04.524 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-21T22:12:04.146210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001510"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:04.526 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:04.703 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:04.703 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:04.711 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-21T22:12:04.146210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001510"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:04.713 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:04.713 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:04.713 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:04.713 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:04.714 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:04.934 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:04.934 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":429,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:04.944 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:04.944 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:05.120 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:05.120 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:05.128 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-21T22:12:04.146210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001510"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:08.129 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:08.308 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:08.309 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:08.317 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2026-02-21T22:12:08.146723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:08.318 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:08.493 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:08.494 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:08.501 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2026-02-21T22:12:08.146723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:08.503 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:08.503 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:08.503 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:08.503 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:08.504 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:08.720 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:08.721 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":433,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:08.729 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:08.729 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:08.907 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:08.907 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:08.915 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2026-02-21T22:12:08.146723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:11.918 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:12.096 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:12.096 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:12.105 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2026-02-21T22:12:10.146905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001396"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:12.106 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:12.281 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:12.281 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:12.289 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2026-02-21T22:12:12.147149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:12.290 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:12.290 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:12.290 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:12.290 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:12.290 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:12.508 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:12.508 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:12.518 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:12.518 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:12.694 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:12.694 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:12.703 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2026-02-21T22:12:12.147149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:15.704 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:15.878 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:15.878 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:15.886 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2026-02-21T22:12:14.147386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001452"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:15.886 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:16.061 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:16.061 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:16.069 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2026-02-21T22:12:14.147386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001452"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:16.071 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:16.071 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:16.071 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:16.071 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:16.072 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:16.294 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:16.294 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:16.304 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:16.305 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:16.481 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:16.482 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:16.489 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2026-02-21T22:12:16.147674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001463"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:19.492 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:19.667 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:19.667 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:19.675 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2026-02-21T22:12:18.147906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001454"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:19.676 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:19.850 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:19.850 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:19.858 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2026-02-21T22:12:18.147906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001454"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:19.859 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:19.859 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:19.859 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:19.859 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:19.860 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:20.075 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:20.075 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":444,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:20.084 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:20.085 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:20.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:20.258 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:20.267 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2026-02-21T22:12:20.148163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:23.268 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:23.444 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:23.444 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:23.452 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-21T22:12:22.148397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:23.453 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:23.629 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:23.629 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:23.637 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-21T22:12:22.148397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:23.638 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:23.638 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:23.638 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:23.638 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:23.638 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:23.855 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:23.855 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:23.863 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:23.864 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:24.043 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:24.044 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:24.053 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-21T22:12:22.148397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:27.056 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:27.236 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:27.236 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:27.245 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2026-02-21T22:12:26.148998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001611"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:27.246 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:27.427 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:27.427 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:27.436 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2026-02-21T22:12:26.148998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001611"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:27.438 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:27.438 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:27.438 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:27.438 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:27.439 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:27.655 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:27.655 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":452,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:27.664 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:27.664 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:27.840 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:27.841 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:27.849 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2026-02-21T22:12:26.148998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001611"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:30.850 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:31.032 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:31.033 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:31.043 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2026-02-21T22:12:30.149522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:31.044 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:31.221 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:31.222 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:31.231 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2026-02-21T22:12:30.149522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:31.232 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:31.232 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:31.232 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:31.232 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:31.232 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:31.454 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:31.454 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":456,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:31.463 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:31.464 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:31.640 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:31.640 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:31.648 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2026-02-21T22:12:30.149522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:34.649 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:34.825 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:34.826 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:34.833 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-21T22:12:34.150037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001639"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:34.835 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:35.010 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:35.010 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:35.019 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-21T22:12:34.150037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001639"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:35.019 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:35.019 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:35.019 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:35.020 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:35.020 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:35.234 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:35.235 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":460,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:35.244 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:35.244 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:35.418 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:35.419 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:35.427 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-21T22:12:34.150037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001639"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:38.428 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:38.605 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:38.605 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:38.613 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2026-02-21T22:12:38.150615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:38.614 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:38.788 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:38.789 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:38.797 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2026-02-21T22:12:38.150615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:38.798 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:38.798 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:38.798 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:38.798 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:38.798 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:39.019 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:39.020 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":463,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:39.029 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:39.029 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:39.206 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:39.206 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:39.215 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2026-02-21T22:12:38.150615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:42.217 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:42.397 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:42.397 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:42.407 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2026-02-21T22:12:42.151036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001514"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:42.408 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:42.586 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:42.586 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:42.594 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2026-02-21T22:12:42.151036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001514"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:42.596 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:42.596 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:42.596 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:42.596 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:42.597 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:42.813 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:42.813 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":467,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:42.822 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:42.822 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:43.001 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:43.001 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:43.010 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2026-02-21T22:12:42.151036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001514"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:46.012 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:46.192 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:46.192 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:46.200 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2026-02-21T22:12:46.151565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:46.201 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:46.377 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:46.378 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:46.386 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2026-02-21T22:12:46.151565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:46.387 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:46.387 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:46.387 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:46.387 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:46.388 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:46.608 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:46.608 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":471,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:46.617 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:46.618 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:46.797 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:46.797 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:46.805 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2026-02-21T22:12:46.151565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:49.808 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:49.984 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:49.984 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:49.992 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2026-02-21T22:12:48.151811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001509"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:49.992 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:50.168 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:50.169 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:50.177 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2026-02-21T22:12:50.152082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001467"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:50.179 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:50.179 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:50.179 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:50.179 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:50.180 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:50.397 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:50.397 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":475,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:50.406 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:50.406 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:50.585 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:50.586 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:50.594 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2026-02-21T22:12:50.152082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001467"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:53.596 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:53.772 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:53.773 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:53.780 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2026-02-21T22:12:52.152295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001465"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:53.782 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:53.956 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:53.957 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:53.965 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2026-02-21T22:12:52.152295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001465"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:53.966 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:53.966 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:53.967 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:53.967 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:53.967 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:54.184 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:54.184 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":479,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:54.192 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:54.192 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:54.367 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:54.368 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:54.376 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2026-02-21T22:12:54.152579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001542"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:57.379 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:57.557 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:57.557 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:57.565 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2026-02-21T22:12:56.152810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001520"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:57.567 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:57.748 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:57.748 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:57.757 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2026-02-21T22:12:56.152810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001520"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:12:57.758 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:12:57.758 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:12:57.759 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:12:57.759 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:12:57.759 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:12:57.977 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:57.977 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":482,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:12:57.987 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:12:57.987 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:12:58.166 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:12:58.166 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:12:58.175 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2026-02-21T22:12:58.153070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:01.177 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:01.354 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:01.355 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:01.363 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2026-02-21T22:13:00.153298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001486"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:01.364 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:01.542 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:01.542 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:01.551 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2026-02-21T22:13:00.153298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001486"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:01.552 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:01.552 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:01.552 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:01.552 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:01.552 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:01.768 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:01.769 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":486,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:01.777 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:01.778 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:01.954 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:01.955 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:01.963 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2026-02-21T22:13:00.153298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001486"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:04.965 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:05.143 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:05.143 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:05.152 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2026-02-21T22:13:04.153842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001546"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:05.153 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:05.333 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:05.334 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:05.343 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2026-02-21T22:13:04.153842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001546"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:05.344 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:05.344 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:05.344 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:05.344 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:05.345 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:05.565 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:05.566 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:05.574 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:05.574 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:05.750 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:05.751 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:05.759 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2026-02-21T22:13:04.153842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001546"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:08.761 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:08.936 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:08.937 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:08.944 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2026-02-21T22:13:08.154338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:08.945 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:09.121 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:09.121 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:09.129 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2026-02-21T22:13:08.154338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:09.130 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:09.130 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:09.130 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:09.130 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:09.130 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:09.346 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:09.346 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:09.355 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:09.355 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:09.531 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:09.531 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:09.539 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2026-02-21T22:13:08.154338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:12.540 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:12.716 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:12.716 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:12.724 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2026-02-21T22:13:12.154778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001480"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:12.726 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:12.901 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:12.902 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:12.909 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2026-02-21T22:13:12.154778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001480"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:12.909 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:12.910 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:12.910 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:12.910 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:12.910 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:13.122 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:13.123 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:13.131 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:13.131 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:13.306 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:13.306 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:13.314 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2026-02-21T22:13:12.154778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001480"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209227+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:09:15.209227+0000","last_peered":"2026-02-21T22:09:15.209227+0000","last_clean":"2026-02-21T22:09:15.209227+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:09:15.209227+0000","last_undegraded":"2026-02-21T22:09:15.209227+0000","last_fullsized":"2026-02-21T22:09:15.209227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:16.316 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:16.494 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:16.494 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:16.503 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2026-02-21T22:13:16.155289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:16.504 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:16.679 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:16.679 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:16.687 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2026-02-21T22:13:16.155289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:16.688 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:16.688 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:16.688 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:16.688 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:16.688 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:16.907 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:16.907 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":501,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:16.916 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:16.916 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:17.093 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:17.093 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:17.101 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2026-02-21T22:13:16.155289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:20.103 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:20.283 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:20.283 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:20.292 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-21T22:13:20.155700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001362"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:20.293 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:20.470 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:20.471 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:20.480 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-21T22:13:20.155700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001362"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:20.481 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:20.481 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:20.482 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:20.482 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:20.482 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:20.699 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:20.699 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":505,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:20.709 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:20.709 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:20.888 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:20.888 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:20.896 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-21T22:13:20.155700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001362"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:23.899 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:24.078 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:24.079 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:24.086 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2026-02-21T22:13:22.155924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001361"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:24.087 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:24.262 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:24.262 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:24.272 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2026-02-21T22:13:24.156192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:24.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:24.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:24.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:24.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:24.272 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:24.491 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:24.491 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":509,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:24.500 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:24.500 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:24.676 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:24.677 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:24.685 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2026-02-21T22:13:24.156192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209253+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:09:15.209253+0000","last_peered":"2026-02-21T22:09:15.209253+0000","last_clean":"2026-02-21T22:09:15.209253+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:09:15.209253+0000","last_undegraded":"2026-02-21T22:09:15.209253+0000","last_fullsized":"2026-02-21T22:09:15.209253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209625+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:09:15.209625+0000","last_peered":"2026-02-21T22:09:15.209625+0000","last_clean":"2026-02-21T22:09:15.209625+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:09:15.209625+0000","last_undegraded":"2026-02-21T22:09:15.209625+0000","last_fullsized":"2026-02-21T22:09:15.209625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:15.209659+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:09:15.209659+0000","last_peered":"2026-02-21T22:09:15.209659+0000","last_clean":"2026-02-21T22:09:15.209659+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:09:15.209659+0000","last_undegraded":"2026-02-21T22:09:15.209659+0000","last_fullsized":"2026-02-21T22:09:15.209659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:27.688 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:27.864 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:27.865 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:27.873 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2026-02-21T22:13:26.156389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001336"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:27.875 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:28.050 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:28.051 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:28.059 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2026-02-21T22:13:26.156389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001336"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:28.061 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:28.061 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:28.061 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:28.061 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:28.062 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:28.276 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:28.276 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":513,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:28.285 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:28.285 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:28.460 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:28.460 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:28.468 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2026-02-21T22:13:28.156565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001276"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:31.470 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:31.645 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:31.645 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:31.653 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2026-02-21T22:13:30.156767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:31.654 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:31.831 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:31.831 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:31.839 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2026-02-21T22:13:30.156767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:20.209951+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:09:20.209951+0000","last_peered":"2026-02-21T22:09:20.209951+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:09:20.209951+0000","last_undegraded":"2026-02-21T22:09:20.209951+0000","last_fullsized":"2026-02-21T22:09:20.209951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:31.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:31.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:31.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:31.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:20.209951+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:09:20.209951+0000', 'last_peered': '2026-02-21T22:09:20.209951+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:09:20.209951+0000', 'last_undegraded': '2026-02-21T22:09:20.209951+0000', 'last_fullsized': '2026-02-21T22:09:20.209951+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:31.840 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:32.059 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:32.059 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:32.069 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:32.070 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:32.249 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:32.250 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:32.260 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2026-02-21T22:13:32.156967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001267"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:35.262 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:35.441 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:35.441 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:35.450 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2026-02-21T22:13:34.157205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001280"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:35.452 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:35.630 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:35.630 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:35.639 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2026-02-21T22:13:34.157205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001280"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:35.640 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:35.640 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:09:25.209793+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:09:25.209793+0000', 'last_peered': '2026-02-21T22:09:25.209793+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:09:25.209793+0000', 'last_undegraded': '2026-02-21T22:09:25.209793+0000', 'last_fullsized': '2026-02-21T22:09:25.209793+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:35.641 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:35.641 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:35.641 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:35.859 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:35.859 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":520,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:35.868 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:35.868 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:36.048 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:36.048 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:36.056 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2026-02-21T22:13:34.157205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001280"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:09:25.209793+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:09:25.209793+0000","last_peered":"2026-02-21T22:09:25.209793+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:09:25.209793+0000","last_undegraded":"2026-02-21T22:09:25.209793+0000","last_fullsized":"2026-02-21T22:09:25.209793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:39.057 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:39.235 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:39.235 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:39.243 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2026-02-21T22:13:38.157676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001286"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:39.245 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:39.423 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:39.423 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:39.432 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2026-02-21T22:13:38.157676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001286"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:39.433 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:39.433 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:39.433 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:39.433 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:39.433 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:39.651 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:39.652 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":524,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:39.660 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:39.660 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:39.835 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:39.836 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:39.844 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2026-02-21T22:13:38.157676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001286"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:42.846 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:43.022 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:43.022 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:43.031 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2026-02-21T22:13:42.158088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:43.033 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:43.209 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:43.209 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:43.218 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2026-02-21T22:13:42.158088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:43.219 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:43.219 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:43.219 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:43.219 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:43.220 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:43.434 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:43.434 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":528,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:43.442 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:43.443 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:43.617 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:43.617 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:43.625 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2026-02-21T22:13:42.158088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:25.210287+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:09:25.210287+0000","last_peered":"2026-02-21T22:09:25.210287+0000","last_clean":"2026-02-21T22:09:25.210287+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:09:25.210287+0000","last_undegraded":"2026-02-21T22:09:25.210287+0000","last_fullsized":"2026-02-21T22:09:25.210287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:46.627 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:46.803 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:46.804 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:46.812 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2026-02-21T22:13:46.158572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:46.813 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:46.988 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:46.989 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:46.998 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2026-02-21T22:13:46.158572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:46.999 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:46.999 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:47.000 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:47.000 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:47.000 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:47.219 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:47.219 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":532,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:47.228 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:47.229 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:47.406 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:47.406 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:47.414 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2026-02-21T22:13:46.158572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:50.415 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:50.592 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:50.593 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:50.601 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2026-02-21T22:13:50.159088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:50.603 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:50.782 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:50.782 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:50.790 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2026-02-21T22:13:50.159088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:50.791 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:50.792 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:50.792 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:50.792 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:50.792 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:51.011 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:51.011 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":535,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:51.020 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:51.020 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:51.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:51.198 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:51.207 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2026-02-21T22:13:50.159088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:09:30.210792+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:09:30.210792+0000","last_peered":"2026-02-21T22:09:30.210792+0000","last_clean":"2026-02-21T22:09:30.210792+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:09:30.210792+0000","last_undegraded":"2026-02-21T22:09:30.210792+0000","last_fullsized":"2026-02-21T22:09:30.210792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:54.208 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:54.385 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:54.385 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:54.394 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2026-02-21T22:13:54.159616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:54.395 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:54.572 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:54.572 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:54.581 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2026-02-21T22:13:54.159616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:54.581 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:54.581 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:54.581 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:54.581 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:54.581 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:54.800 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:54.800 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":539,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:54.809 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:54.809 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:54.984 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:54.984 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:54.992 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2026-02-21T22:13:54.159616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:57.993 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:58.170 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:58.170 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:58.178 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-21T22:13:58.160128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001555"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:58.179 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:58.357 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:58.357 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:58.366 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-21T22:13:58.160128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001555"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:13:58.367 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:13:58.367 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:13:58.368 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:13:58.368 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:13:58.368 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:13:58.586 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:58.587 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":543,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:13:58.596 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:13:58.596 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:13:58.774 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:13:58.775 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:13:58.783 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-21T22:13:58.160128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001555"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:01.785 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:01.962 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:01.962 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:01.971 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2026-02-21T22:14:00.160351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001531"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:01.972 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:02.146 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:02.146 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:02.154 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2026-02-21T22:14:00.160351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001531"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:02.154 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:02.154 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:02.154 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:02.154 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:02.155 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:02.371 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:02.371 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:02.381 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:02.381 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:02.555 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:02.556 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:02.565 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2026-02-21T22:14:02.160616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:05.566 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:05.746 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:05.746 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:05.753 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2026-02-21T22:14:04.160833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:05.755 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:05.929 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:05.930 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:05.938 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2026-02-21T22:14:04.160833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:05.938 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:05.938 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:05.938 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:05.938 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:05.938 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:06.156 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:06.156 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":550,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:06.167 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:06.167 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:06.343 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:06.344 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:06.352 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2026-02-21T22:14:06.161090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001473"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:09.353 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:09.531 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:09.532 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:09.540 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2026-02-21T22:14:08.161329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:09.542 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:09.719 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:09.719 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:09.727 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2026-02-21T22:14:08.161329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:09.727 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:09.727 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:09.727 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:09.727 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:09.728 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:09.944 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:09.945 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:09.953 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:09.954 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:10.129 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:10.130 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:10.138 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2026-02-21T22:14:08.161329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:13.140 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:13.318 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:13.319 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:13.327 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2026-02-21T22:14:12.161759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:13.329 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:13.509 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:13.510 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:13.519 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2026-02-21T22:14:12.161759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:13.520 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:13.521 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:13.521 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:13.521 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:13.521 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:13.741 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:13.742 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":558,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:13.750 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:13.750 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:13.928 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:13.928 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:13.938 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2026-02-21T22:14:12.161759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:16.939 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:17.115 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:17.115 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:17.123 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2026-02-21T22:14:16.162246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:17.123 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:17.304 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:17.305 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:17.314 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2026-02-21T22:14:16.162246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:17.315 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:17.315 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:17.316 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:17.316 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:17.316 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:17.532 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:17.532 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:17.541 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:17.541 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:17.720 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:17.720 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:17.729 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2026-02-21T22:14:16.162246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:20.732 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:20.911 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:20.911 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:20.919 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2026-02-21T22:14:20.162695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001365"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:20.921 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:21.101 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:21.101 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:21.110 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2026-02-21T22:14:20.162695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001365"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:21.113 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:21.113 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:21.114 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:21.114 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:21.115 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:21.332 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:21.333 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":566,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:21.343 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:21.343 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:21.520 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:21.520 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:21.528 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2026-02-21T22:14:20.162695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001365"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:24.530 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:24.706 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:24.707 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:24.715 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2026-02-21T22:14:24.163204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:24.716 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:24.891 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:24.891 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:24.900 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2026-02-21T22:14:24.163204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:24.901 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:24.901 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:24.901 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:24.902 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:24.902 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:25.118 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:25.118 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":569,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:25.128 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:25.128 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:25.305 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:25.305 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:25.314 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2026-02-21T22:14:24.163204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:28.316 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:28.493 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:28.493 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:28.501 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2026-02-21T22:14:28.163771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:28.502 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:28.679 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:28.679 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:28.687 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2026-02-21T22:14:28.163771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:28.688 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:28.688 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:28.688 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:28.688 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:28.688 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:28.906 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:28.906 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":573,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:28.916 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:28.916 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:29.094 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:29.095 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:29.103 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2026-02-21T22:14:28.163771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:32.105 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:32.283 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:32.283 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:32.291 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-21T22:14:32.164288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001592"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:32.292 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:32.468 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:32.469 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:32.478 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-21T22:14:32.164288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001592"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:32.479 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:32.479 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:32.480 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:32.480 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:32.480 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:32.698 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:32.698 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":577,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:32.707 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:32.707 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:32.883 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:32.884 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:32.892 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-21T22:14:32.164288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001592"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:35.894 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:36.073 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:36.074 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:36.082 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2026-02-21T22:14:34.164532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001614"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:36.084 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:36.263 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:36.263 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:36.273 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2026-02-21T22:14:36.164796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001591"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:36.274 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:36.274 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:36.274 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:36.275 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:36.275 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:36.497 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:36.497 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":581,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:36.507 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:36.508 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:36.689 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:36.689 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:36.698 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2026-02-21T22:14:36.164796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001591"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:39.699 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:39.879 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:39.879 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:39.888 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2026-02-21T22:14:38.165027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001563"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:39.889 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:40.064 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:40.064 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:40.073 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2026-02-21T22:14:38.165027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001563"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:40.074 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:40.074 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:40.074 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:40.074 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:40.074 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:40.293 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:40.293 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":585,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:40.303 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:40.304 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:40.482 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:40.483 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:40.491 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2026-02-21T22:14:40.165244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001473"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:43.494 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:43.669 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:43.670 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:43.677 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2026-02-21T22:14:42.165450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:43.678 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:43.851 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:43.851 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:43.860 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2026-02-21T22:14:42.165450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:43.860 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:43.860 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:43.860 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:43.860 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:43.860 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:44.074 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:44.074 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":588,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:44.083 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:44.083 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:44.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:44.258 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:44.267 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2026-02-21T22:14:44.165696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001408"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:47.269 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:47.445 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:47.445 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:47.453 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2026-02-21T22:14:46.165889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:47.455 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:47.629 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:47.629 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:47.638 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2026-02-21T22:14:46.165889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:47.640 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:47.640 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:47.640 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:47.640 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:47.640 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:47.855 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:47.856 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":592,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:47.864 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:47.864 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:48.038 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:48.039 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:48.047 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2026-02-21T22:14:46.165889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:51.048 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:51.223 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:51.223 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:51.232 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2026-02-21T22:14:50.166286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001258"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:51.232 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:51.410 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:51.410 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:51.420 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2026-02-21T22:14:50.166286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001258"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:51.421 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:51.422 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:51.422 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:51.422 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:51.422 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:51.639 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:51.639 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":596,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:51.648 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:51.648 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:51.823 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:51.824 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:51.832 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2026-02-21T22:14:50.166286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001258"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:54.835 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:55.013 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:55.014 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:55.022 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2026-02-21T22:14:54.166790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:55.024 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:55.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:55.198 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:55.208 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2026-02-21T22:14:54.166790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:55.208 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:55.208 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:55.208 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:55.208 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:55.208 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:55.424 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:55.424 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":600,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:55.432 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:55.433 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:55.610 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:55.610 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:55.617 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2026-02-21T22:14:54.166790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:58.618 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:58.794 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:58.795 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:58.803 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2026-02-21T22:14:58.167233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:58.805 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:58.979 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:58.980 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:58.987 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2026-02-21T22:14:58.167233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:14:58.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:14:58.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:14:58.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:14:58.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:14:58.990 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:14:59.205 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:59.206 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":604,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:14:59.214 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:14:59.215 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:14:59.391 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:14:59.392 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:14:59.400 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2026-02-21T22:14:58.167233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:02.402 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:02.576 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:02.576 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:02.584 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2026-02-21T22:15:02.167721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:02.586 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:02.762 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:02.763 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:02.770 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2026-02-21T22:15:02.167721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:02.772 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:02.772 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:02.772 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:02.772 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:02.773 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:02.989 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:02.989 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":607,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:02.998 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:02.999 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:03.174 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:03.175 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:03.183 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2026-02-21T22:15:02.167721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:06.186 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:06.363 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:06.363 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:06.371 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2026-02-21T22:15:06.168199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:06.373 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:06.547 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:06.547 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:06.555 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2026-02-21T22:15:06.168199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:06.556 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:06.556 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:06.557 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:06.557 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:06.557 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:06.775 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:06.775 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":611,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:06.785 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:06.785 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:06.958 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:06.958 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:06.966 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2026-02-21T22:15:06.168199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:09.967 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:10.143 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:10.144 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:10.151 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2026-02-21T22:15:08.168409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:10.152 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:10.329 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:10.329 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:10.337 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2026-02-21T22:15:10.168691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:10.338 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:10.339 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:10.339 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:10.339 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:10.339 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:10.554 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:10.554 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":615,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:10.563 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:10.563 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:10.736 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:10.736 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:10.744 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2026-02-21T22:15:10.168691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:13.746 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:13.925 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:13.925 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:13.933 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2026-02-21T22:15:12.168894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:13.935 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:14.109 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:14.110 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:14.117 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2026-02-21T22:15:12.168894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:14.119 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:14.119 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:14.119 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:14.119 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:14.120 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:14.338 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:14.338 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":619,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:14.347 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:14.347 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:14.522 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:14.523 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:14.530 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2026-02-21T22:15:14.169116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:15.218419+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:11:15.218419+0000","last_peered":"2026-02-21T22:11:15.218419+0000","last_clean":"2026-02-21T22:11:15.218419+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:11:15.218419+0000","last_undegraded":"2026-02-21T22:11:15.218419+0000","last_fullsized":"2026-02-21T22:11:15.218419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:17.532 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:17.709 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:17.710 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:17.718 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2026-02-21T22:15:16.169317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001360"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:17.719 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:17.894 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:17.894 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:17.903 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2026-02-21T22:15:16.169317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001360"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:17.904 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:17.904 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:17.904 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:17.905 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:17.905 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:18.121 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:18.121 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":622,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:18.130 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:18.130 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:18.306 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:18.306 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:18.315 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2026-02-21T22:15:18.169531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001331"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:21.317 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:21.491 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:21.492 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:21.499 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2026-02-21T22:15:20.169747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001338"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:21.501 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:21.675 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:21.675 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:21.684 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2026-02-21T22:15:20.169747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001338"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:21.684 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:21.684 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:21.684 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:21.684 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:21.684 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:21.899 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:21.899 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":626,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:21.907 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:21.908 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:22.081 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:22.081 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:22.089 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2026-02-21T22:15:20.169747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001338"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:25.092 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:25.269 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:25.269 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:25.278 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2026-02-21T22:15:24.170199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001304"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:25.280 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:25.457 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:25.458 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:25.466 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2026-02-21T22:15:24.170199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001304"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:25.467 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:25.467 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:25.467 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:25.468 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:25.468 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:25.686 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:25.687 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:25.697 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:25.697 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:25.874 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:25.875 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:25.883 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2026-02-21T22:15:24.170199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001304"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:28.885 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:29.059 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:29.059 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:29.067 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2026-02-21T22:15:28.170634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001317"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:29.068 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:29.244 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:29.245 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:29.254 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2026-02-21T22:15:28.170634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001317"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:29.255 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:29.255 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:29.255 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:29.255 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:29.256 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:29.472 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:29.472 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:29.481 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:29.481 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:29.658 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:29.658 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:29.666 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2026-02-21T22:15:28.170634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001317"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218722+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:11:20.218722+0000","last_peered":"2026-02-21T22:11:20.218722+0000","last_clean":"2026-02-21T22:11:20.218722+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:11:20.218722+0000","last_undegraded":"2026-02-21T22:11:20.218722+0000","last_fullsized":"2026-02-21T22:11:20.218722+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218743+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:11:20.218743+0000","last_peered":"2026-02-21T22:11:20.218743+0000","last_clean":"2026-02-21T22:11:20.218743+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:11:20.218743+0000","last_undegraded":"2026-02-21T22:11:20.218743+0000","last_fullsized":"2026-02-21T22:11:20.218743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:20.218765+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:11:20.218765+0000","last_peered":"2026-02-21T22:11:20.218765+0000","last_clean":"2026-02-21T22:11:20.218765+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:11:20.218765+0000","last_undegraded":"2026-02-21T22:11:20.218765+0000","last_fullsized":"2026-02-21T22:11:20.218765+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:32.669 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:32.843 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:32.844 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:32.851 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2026-02-21T22:15:32.171073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001326"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:32.852 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:33.026 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:33.026 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:33.034 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2026-02-21T22:15:32.171073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001326"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:33.035 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:33.035 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:33.035 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:33.036 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:25.219048+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:11:25.219048+0000', 'last_peered': '2026-02-21T22:11:25.219048+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:11:25.219048+0000', 'last_undegraded': '2026-02-21T22:11:25.219048+0000', 'last_fullsized': '2026-02-21T22:11:25.219048+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:33.036 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:33.258 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:33.258 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":638,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:33.267 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:33.268 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:33.446 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:33.446 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:33.456 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2026-02-21T22:15:32.171073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001326"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:25.219048+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:11:25.219048+0000","last_peered":"2026-02-21T22:11:25.219048+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:11:25.219048+0000","last_undegraded":"2026-02-21T22:11:25.219048+0000","last_fullsized":"2026-02-21T22:11:25.219048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:36.458 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:36.637 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:36.637 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:36.646 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2026-02-21T22:15:36.171523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001324"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:36.647 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:36.822 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:36.823 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:36.830 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2026-02-21T22:15:36.171523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001324"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:36.832 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:36.832 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:36.832 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:36.832 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:36.833 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":641,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:37.058 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:37.059 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:37.242 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2026-02-21T22:15:36.171523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001324"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:40.244 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:40.418 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-21T22:15:40.172063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:40.428 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:40.603 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:40.603 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:40.611 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-21T22:15:40.172063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:40.613 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:40.613 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:11:30.219429+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:11:30.219429+0000', 'last_peered': '2026-02-21T22:11:30.219429+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:11:30.219429+0000', 'last_undegraded': '2026-02-21T22:11:30.219429+0000', 'last_fullsized': '2026-02-21T22:11:30.219429+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:40.613 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:40.613 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:40.613 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:40.831 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:40.831 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":645,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:40.840 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:40.840 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:41.018 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:41.019 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:41.027 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-21T22:15:40.172063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:11:30.219429+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:11:30.219429+0000","last_peered":"2026-02-21T22:11:30.219429+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:11:30.219429+0000","last_undegraded":"2026-02-21T22:11:30.219429+0000","last_fullsized":"2026-02-21T22:11:30.219429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:44.030 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:44.209 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:44.210 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:44.218 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2026-02-21T22:15:44.172571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001497"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:44.220 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:44.399 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:44.399 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:44.407 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2026-02-21T22:15:44.172571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001497"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:44.409 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:44.409 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:44.409 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:44.409 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:44.410 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:44.625 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:44.626 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:44.634 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:44.634 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:44.808 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:44.808 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:44.816 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2026-02-21T22:15:44.172571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001497"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:47.819 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:47.996 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:47.996 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:48.004 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2026-02-21T22:15:46.172790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001460"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:48.005 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:48.181 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:48.182 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:48.191 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2026-02-21T22:15:48.173004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001480"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:48.193 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:48.193 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:48.193 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:48.193 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:48.194 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:48.409 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:48.410 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":653,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:48.419 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:48.419 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:48.596 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:48.596 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:48.604 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2026-02-21T22:15:48.173004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001480"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:51.606 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:51.785 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:51.785 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:51.793 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2026-02-21T22:15:50.173224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:51.795 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:51.967 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:51.968 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:51.975 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2026-02-21T22:15:50.173224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:51.976 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:51.977 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:51.977 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:51.977 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:51.977 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:52.194 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:52.194 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":657,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:52.203 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:52.203 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:52.379 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:52.379 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:52.388 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2026-02-21T22:15:52.173464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:55.390 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:55.564 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:55.564 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:55.573 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2026-02-21T22:15:54.173716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:55.574 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:55.747 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:55.748 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:55.756 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2026-02-21T22:15:54.173716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:55.757 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:55.757 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:55.757 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:55.757 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:55.758 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:55.972 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:55.972 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":660,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:55.980 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:55.981 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:56.156 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:56.156 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:56.164 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2026-02-21T22:15:54.173716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:35.219842+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:11:35.219842+0000","last_peered":"2026-02-21T22:11:35.219842+0000","last_clean":"2026-02-21T22:11:35.219842+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:11:35.219842+0000","last_undegraded":"2026-02-21T22:11:35.219842+0000","last_fullsized":"2026-02-21T22:11:35.219842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:59.166 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:59.343 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:59.343 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:59.352 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2026-02-21T22:15:58.174161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001371"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:59.353 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:59.529 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:59.529 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:59.537 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2026-02-21T22:15:58.174161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001371"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:59.539 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:15:59.539 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:15:59.539 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:15:59.539 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:15:59.540 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:15:59.759 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:59.759 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":664,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:15:59.768 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:15:59.769 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:59.944 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:15:59.945 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:15:59.952 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2026-02-21T22:15:58.174161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001371"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:11:40.220104+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:11:40.220104+0000","last_peered":"2026-02-21T22:11:40.220104+0000","last_clean":"2026-02-21T22:11:40.220104+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:11:40.220104+0000","last_undegraded":"2026-02-21T22:11:40.220104+0000","last_fullsized":"2026-02-21T22:11:40.220104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:02.954 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:03.132 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:03.133 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:03.140 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2026-02-21T22:16:02.174646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:03.142 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:03.321 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:03.321 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:03.330 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2026-02-21T22:16:02.174646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:03.332 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:03.332 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:03.332 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:03.332 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:03.333 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:03.550 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:03.550 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":668,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:03.558 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:03.558 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:03.735 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:03.735 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:03.743 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2026-02-21T22:16:02.174646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:06.745 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:06.920 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:06.920 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:06.928 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2026-02-21T22:16:06.175093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001377"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:06.930 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:07.107 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:07.107 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:07.115 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2026-02-21T22:16:06.175093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001377"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:07.116 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:07.116 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:07.117 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:07.117 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:07.117 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:07.337 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:07.337 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":672,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:07.347 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:07.347 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:07.522 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:07.523 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:07.530 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2026-02-21T22:16:06.175093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001377"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:10.532 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:10.709 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:10.710 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:10.717 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2026-02-21T22:16:10.175591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:10.719 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:10.894 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:10.894 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:10.903 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2026-02-21T22:16:10.175591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:10.904 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:10.904 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:10.904 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:10.904 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:10.905 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:11.120 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:11.121 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":675,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:11.129 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:11.129 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:11.304 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:11.304 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:11.313 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2026-02-21T22:16:10.175591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:14.315 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:14.488 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:14.489 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:14.497 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2026-02-21T22:16:14.176113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001467"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:14.498 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:14.678 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:14.679 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:14.687 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2026-02-21T22:16:14.176113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001467"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:14.688 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:14.689 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:14.689 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:14.689 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:14.689 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:14.910 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:14.910 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:14.919 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:14.919 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:15.095 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:15.096 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:15.103 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2026-02-21T22:16:14.176113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001467"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:18.104 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:18.283 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:18.283 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:18.291 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2026-02-21T22:16:18.176542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:18.293 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:18.472 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:18.473 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:18.482 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2026-02-21T22:16:18.176542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:18.483 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:18.483 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:18.483 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:18.484 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:18.484 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:18.701 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:18.701 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":683,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:18.711 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:18.711 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:18.887 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:18.887 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:18.895 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2026-02-21T22:16:18.176542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:21.898 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:22.074 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:22.075 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:22.082 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2026-02-21T22:16:20.176734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:22.083 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:22.257 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:22.258 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:22.265 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2026-02-21T22:16:22.176982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001391"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:22.267 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:22.267 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:22.267 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:22.267 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:22.268 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:22.488 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:22.488 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":687,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:22.496 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:22.497 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:22.668 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:22.668 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:22.677 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2026-02-21T22:16:22.176982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001391"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:25.680 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:25.856 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:25.857 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:25.864 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2026-02-21T22:16:24.177198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:25.866 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:26.041 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:26.041 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:26.049 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2026-02-21T22:16:24.177198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:26.050 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:26.050 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:26.050 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:26.050 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:26.051 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:26.268 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:26.268 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":691,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:26.278 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:26.278 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:26.452 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:26.453 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:26.461 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2026-02-21T22:16:26.177413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001300"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:29.462 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:29.637 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:29.637 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:29.645 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2026-02-21T22:16:28.177644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001307"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:29.646 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:29.820 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:29.821 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:29.830 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2026-02-21T22:16:28.177644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001307"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:29.831 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:29.831 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:29.832 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:29.832 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:29.832 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:30.047 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:30.048 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:30.056 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:30.056 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:30.230 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:30.230 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:30.239 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2026-02-21T22:16:30.177907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001365"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:33.241 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:33.420 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:33.420 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:33.429 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2026-02-21T22:16:32.178114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:33.430 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:33.608 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:33.608 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:33.617 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2026-02-21T22:16:32.178114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:33.618 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:33.618 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:33.619 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:33.619 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:33.619 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:33.836 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:33.836 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:33.845 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:33.845 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:34.019 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:34.020 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:34.028 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2026-02-21T22:16:32.178114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:37.030 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:37.204 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:37.204 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:37.213 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2026-02-21T22:16:36.178613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:37.214 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:37.388 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:37.388 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:37.397 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2026-02-21T22:16:36.178613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:37.399 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:37.399 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:37.399 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:37.399 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:37.400 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:37.616 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:37.617 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":702,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:37.626 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:37.626 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:37.802 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:37.802 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:37.811 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2026-02-21T22:16:36.178613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:40.813 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:40.990 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:40.990 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:40.998 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2026-02-21T22:16:40.179166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001521"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:40.999 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:41.175 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:41.176 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:41.184 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2026-02-21T22:16:40.179166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001521"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:41.185 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:41.185 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:41.186 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:41.186 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:41.186 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:41.405 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:41.405 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":706,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:41.415 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:41.415 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:41.591 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:41.592 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:41.600 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2026-02-21T22:16:40.179166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001521"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:44.602 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:44.779 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:44.779 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:44.788 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2026-02-21T22:16:44.179709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001595"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:44.789 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:44.964 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:44.964 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:44.972 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2026-02-21T22:16:44.179709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001595"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:44.973 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:44.973 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:44.974 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:44.974 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:44.974 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:45.191 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:45.191 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":710,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:45.200 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:45.201 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:45.376 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:45.377 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:45.385 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2026-02-21T22:16:44.179709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001595"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:48.386 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:48.563 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:48.563 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:48.571 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2026-02-21T22:16:48.180301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001687"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:48.572 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:48.746 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:48.746 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:48.754 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2026-02-21T22:16:48.180301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001687"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:48.754 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:48.754 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:48.755 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:48.755 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:48.755 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:48.969 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:48.969 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":713,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:48.979 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:48.979 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:49.155 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:49.155 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:49.163 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2026-02-21T22:16:48.180301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001687"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:52.165 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:52.343 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:52.344 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:52.352 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2026-02-21T22:16:52.180769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001602"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:52.353 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:52.528 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:52.529 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:52.537 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2026-02-21T22:16:52.180769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001602"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:52.538 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:52.538 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:52.539 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:52.539 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:52.539 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:52.760 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:52.761 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":717,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:52.771 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:52.771 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:52.947 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:52.948 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:52.956 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2026-02-21T22:16:52.180769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001602"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:55.958 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:56.135 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:56.136 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:56.144 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2026-02-21T22:16:54.180997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001597"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:56.144 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:56.320 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:56.320 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:56.329 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2026-02-21T22:16:56.181256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001546"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:56.330 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:16:56.330 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:16:56.331 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:16:56.331 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:16:56.331 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:16:56.550 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:56.550 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":721,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:16:56.560 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:16:56.560 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:56.735 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:56.736 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:56.743 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2026-02-21T22:16:56.181256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001546"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:59.745 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:16:59.923 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:16:59.924 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:16:59.931 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2026-02-21T22:16:58.181430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:16:59.933 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:00.108 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:00.114 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:00.117 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2026-02-21T22:16:58.181430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:00.118 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:00.118 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:00.118 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:00.118 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:00.118 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:00.347 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:00.347 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:00.356 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:00.356 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:00.531 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:00.532 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:00.539 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2026-02-21T22:17:00.181729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001427"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:03.542 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:03.721 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:03.721 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:03.729 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2026-02-21T22:17:02.181947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:03.730 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:03.903 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:03.904 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:03.911 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2026-02-21T22:17:02.181947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:03.912 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:03.913 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:03.913 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:03.913 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:03.913 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:04.132 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:04.132 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":728,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:04.142 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:04.142 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:04.320 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:04.320 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:04.328 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2026-02-21T22:17:04.182168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:07.331 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:07.508 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:07.509 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:07.517 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2026-02-21T22:17:06.182361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:07.518 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:07.697 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:07.697 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:07.706 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2026-02-21T22:17:06.182361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:07.708 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:07.708 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:07.708 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:07.708 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:07.709 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:07.928 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:07.928 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":732,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:07.938 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:07.938 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:08.112 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:08.113 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:08.120 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2026-02-21T22:17:06.182361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:11.121 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:11.298 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:11.299 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:11.307 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2026-02-21T22:17:10.182839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:11.308 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:11.482 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:11.483 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:11.491 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2026-02-21T22:17:10.182839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:11.492 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:11.493 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:11.493 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:11.493 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:11.493 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:11.712 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:11.712 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":736,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:11.721 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:11.722 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:11.900 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:11.900 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:11.908 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2026-02-21T22:17:10.182839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:14.910 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:15.086 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:15.086 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:15.094 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2026-02-21T22:17:14.183378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:15.095 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:15.271 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:15.271 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:15.280 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2026-02-21T22:17:14.183378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:15.280 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:15.280 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:15.280 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:15.280 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:15.281 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:15.499 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:15.499 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":740,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:15.507 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:15.507 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:15.684 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:15.684 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:15.692 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2026-02-21T22:17:14.183378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:15.226594+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:13:15.226594+0000","last_peered":"2026-02-21T22:13:15.226594+0000","last_clean":"2026-02-21T22:13:15.226594+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:13:15.226594+0000","last_undegraded":"2026-02-21T22:13:15.226594+0000","last_fullsized":"2026-02-21T22:13:15.226594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:18.694 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:18.876 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:18.876 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:18.886 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2026-02-21T22:17:18.183936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001575"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:18.887 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:19.067 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:19.068 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:19.076 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2026-02-21T22:17:18.183936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001575"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:19.078 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:19.078 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:19.078 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:19.078 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:19.078 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:19.296 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:19.296 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":744,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:19.305 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:19.305 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:19.479 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:19.480 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:19.487 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2026-02-21T22:17:18.183936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001575"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:22.489 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:22.667 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:22.667 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:22.676 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2026-02-21T22:17:22.184405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001565"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:22.677 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:22.855 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:22.856 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:22.864 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2026-02-21T22:17:22.184405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001565"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:22.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:22.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:22.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:22.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:22.866 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:23.085 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:23.085 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":747,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:23.095 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:23.095 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:23.272 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:23.272 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:23.281 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2026-02-21T22:17:22.184405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001565"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:26.282 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:26.458 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:26.458 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:26.466 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2026-02-21T22:17:26.184902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:26.467 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:26.642 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:26.643 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:26.651 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2026-02-21T22:17:26.184902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:26.652 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:26.652 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:26.653 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:26.653 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:26.653 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:26.870 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:26.870 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":751,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:26.878 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:26.879 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:27.057 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:27.058 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:27.065 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2026-02-21T22:17:26.184902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:30.068 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:30.246 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:30.247 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:30.256 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2026-02-21T22:17:30.185407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:30.257 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:30.433 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:30.434 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:30.442 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2026-02-21T22:17:30.185407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:30.443 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:30.443 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:30.444 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:30.444 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:30.444 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:30.663 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:30.663 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":755,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:30.673 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:30.673 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:30.849 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:30.849 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:30.857 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2026-02-21T22:17:30.185407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:33.859 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:34.035 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:34.035 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:34.043 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2026-02-21T22:17:32.185597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:34.045 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:34.220 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:34.220 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:34.229 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2026-02-21T22:17:34.185818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:34.230 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:34.231 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:34.231 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:34.231 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:34.231 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:34.450 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:34.451 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":759,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:34.459 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:34.460 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:34.637 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:34.638 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:34.646 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2026-02-21T22:17:34.185818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227358+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:13:25.227358+0000","last_peered":"2026-02-21T22:13:25.227358+0000","last_clean":"2026-02-21T22:13:25.227358+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:13:25.227358+0000","last_undegraded":"2026-02-21T22:13:25.227358+0000","last_fullsized":"2026-02-21T22:13:25.227358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227246+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:13:25.227246+0000","last_peered":"2026-02-21T22:13:25.227246+0000","last_clean":"2026-02-21T22:13:25.227246+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:13:25.227246+0000","last_undegraded":"2026-02-21T22:13:25.227246+0000","last_fullsized":"2026-02-21T22:13:25.227246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:25.227265+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:13:25.227265+0000","last_peered":"2026-02-21T22:13:25.227265+0000","last_clean":"2026-02-21T22:13:25.227265+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:13:25.227265+0000","last_undegraded":"2026-02-21T22:13:25.227265+0000","last_fullsized":"2026-02-21T22:13:25.227265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:37.648 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:37.827 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:37.828 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:37.836 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2026-02-21T22:17:36.186043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:37.838 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:38.014 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:38.014 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:38.023 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2026-02-21T22:17:36.186043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:38.024 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:38.024 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:38.025 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:38.025 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:38.025 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:38.242 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:38.242 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":763,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:38.252 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:38.252 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:38.427 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:38.427 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:38.435 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2026-02-21T22:17:38.186339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:41.438 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:41.614 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:41.615 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:41.623 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2026-02-21T22:17:40.186576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:41.625 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:41.800 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:41.800 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:41.808 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2026-02-21T22:17:40.186576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:30.227547+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:13:30.227547+0000","last_peered":"2026-02-21T22:13:30.227547+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:13:30.227547+0000","last_undegraded":"2026-02-21T22:13:30.227547+0000","last_fullsized":"2026-02-21T22:13:30.227547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:41.809 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:41.809 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:41.810 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:41.810 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:30.227547+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:13:30.227547+0000', 'last_peered': '2026-02-21T22:13:30.227547+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:13:30.227547+0000', 'last_undegraded': '2026-02-21T22:13:30.227547+0000', 'last_fullsized': '2026-02-21T22:13:30.227547+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:41.810 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:42.028 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:42.028 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":766,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:42.037 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:42.037 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:42.214 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:42.214 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:42.222 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2026-02-21T22:17:42.186863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:45.223 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:45.402 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:45.402 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:45.410 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2026-02-21T22:17:44.187093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:45.411 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:45.585 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:45.585 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:45.593 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2026-02-21T22:17:44.187093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:45.594 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:45.594 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:13:35.228134+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:13:35.228134+0000', 'last_peered': '2026-02-21T22:13:35.228134+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:13:35.228134+0000', 'last_undegraded': '2026-02-21T22:13:35.228134+0000', 'last_fullsized': '2026-02-21T22:13:35.228134+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:45.595 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:45.595 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:45.595 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:45.811 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:45.811 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":770,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:45.820 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:45.820 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:45.998 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:45.998 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:46.008 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2026-02-21T22:17:44.187093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:13:35.228134+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:13:35.228134+0000","last_peered":"2026-02-21T22:13:35.228134+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:13:35.228134+0000","last_undegraded":"2026-02-21T22:13:35.228134+0000","last_fullsized":"2026-02-21T22:13:35.228134+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:49.009 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:49.185 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:49.185 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:49.194 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2026-02-21T22:17:48.187544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001501"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:49.194 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:49.367 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:49.368 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2026-02-21T22:17:48.187544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001501"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:49.376 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:49.376 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:49.377 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:49.377 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:49.377 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:49.593 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:49.593 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:49.601 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:49.601 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:49.775 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:49.775 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:49.784 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2026-02-21T22:17:48.187544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001501"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:52.785 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:52.963 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:52.964 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:52.972 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2026-02-21T22:17:52.187960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:52.973 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:53.152 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:53.152 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:53.161 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2026-02-21T22:17:52.187960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:53.163 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:53.163 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:53.163 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:53.163 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:53.164 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:53.382 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:53.382 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:53.391 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:53.392 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:53.565 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:53.565 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:53.573 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2026-02-21T22:17:52.187960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:56.575 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:56.752 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:56.752 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:56.760 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2026-02-21T22:17:56.188428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:56.761 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:56.934 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:56.934 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:56.943 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2026-02-21T22:17:56.188428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:17:56.944 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:17:56.944 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:17:56.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:17:56.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:17:56.945 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:17:57.163 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:57.163 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":781,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:17:57.172 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:17:57.172 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:17:57.348 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:17:57.348 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:17:57.357 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2026-02-21T22:17:56.188428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:00.358 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:00.536 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:00.536 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:00.545 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2026-02-21T22:18:00.188889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:00.546 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:00.720 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:00.720 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:00.728 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2026-02-21T22:18:00.188889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:00.730 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:00.730 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:00.730 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:00.730 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:00.730 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:00.948 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:00.948 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":785,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:00.958 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:00.958 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:01.135 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:01.135 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:01.144 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2026-02-21T22:18:00.188889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:04.146 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:04.324 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:04.325 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:04.333 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2026-02-21T22:18:04.189366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:04.335 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:04.514 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:04.514 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:04.523 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2026-02-21T22:18:04.189366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:04.525 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:04.525 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:04.525 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:04.525 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:04.526 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:04.746 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:04.746 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":789,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:04.756 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:04.756 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:04.933 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:04.934 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:04.942 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2026-02-21T22:18:04.189366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:45.228644+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:13:45.228644+0000","last_peered":"2026-02-21T22:13:45.228644+0000","last_clean":"2026-02-21T22:13:45.228644+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:13:45.228644+0000","last_undegraded":"2026-02-21T22:13:45.228644+0000","last_fullsized":"2026-02-21T22:13:45.228644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:07.944 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:08.122 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:08.123 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:08.131 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2026-02-21T22:18:06.189576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:08.132 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:08.308 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:08.308 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:08.317 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2026-02-21T22:18:08.189872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:08.318 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:08.318 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:08.319 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:08.319 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:08.319 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:08.538 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:08.538 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":793,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:08.547 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:08.548 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:08.723 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:08.723 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:08.731 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2026-02-21T22:18:08.189872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:11.732 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:11.909 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:11.910 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:11.918 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2026-02-21T22:18:10.190090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001472"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:11.919 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:12.094 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:12.095 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:12.103 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2026-02-21T22:18:10.190090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001472"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:13:50.229120+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:13:50.229120+0000","last_peered":"2026-02-21T22:13:50.229120+0000","last_clean":"2026-02-21T22:13:50.229120+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:13:50.229120+0000","last_undegraded":"2026-02-21T22:13:50.229120+0000","last_fullsized":"2026-02-21T22:13:50.229120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:12.104 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:12.104 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:12.104 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:12.104 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:12.105 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:12.322 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:12.323 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":797,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:12.333 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:12.333 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:12.511 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:12.511 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:12.520 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2026-02-21T22:18:12.190339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001450"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:15.523 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:15.705 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:15.705 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:15.713 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2026-02-21T22:18:14.190562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001455"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:15.715 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:15.892 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:15.892 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:15.900 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2026-02-21T22:18:14.190562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001455"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:15.901 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:15.902 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:15.902 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:15.902 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:15.902 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:16.117 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:16.117 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":800,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:16.126 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:16.126 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:16.306 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:16.307 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:16.316 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2026-02-21T22:18:16.190833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001466"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:19.319 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:19.498 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:19.499 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:19.507 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2026-02-21T22:18:18.191043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001466"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:19.508 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:19.681 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:19.681 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:19.689 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2026-02-21T22:18:18.191043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001466"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:19.691 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:19.691 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:19.691 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:19.691 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:19.692 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:19.915 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:19.915 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":804,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:19.924 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:19.924 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:20.098 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:20.098 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:20.106 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2026-02-21T22:18:18.191043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001466"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:23.108 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:23.287 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:23.288 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:23.298 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2026-02-21T22:18:22.191485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:23.300 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:23.475 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:23.475 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:23.483 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2026-02-21T22:18:22.191485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:23.484 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:23.484 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:23.484 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:23.484 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:23.484 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:23.703 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:23.703 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":808,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:23.712 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:23.712 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:23.883 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:23.884 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:23.891 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2026-02-21T22:18:22.191485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001395"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:26.893 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:27.072 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:27.072 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:27.081 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2026-02-21T22:18:26.191981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001418"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:27.082 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:27.260 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:27.260 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:27.268 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2026-02-21T22:18:26.191981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001418"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:27.269 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:27.269 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:27.269 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:27.269 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:27.269 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:27.488 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:27.488 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":812,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:27.496 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:27.496 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:27.671 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:27.671 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:27.679 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2026-02-21T22:18:26.191981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001418"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:30.682 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:30.857 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:30.857 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:30.865 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2026-02-21T22:18:30.192453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:30.866 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:31.044 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:31.044 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:31.053 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2026-02-21T22:18:30.192453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:31.054 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:31.054 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:31.055 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:31.055 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:31.055 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:31.272 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:31.273 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":816,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:31.284 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:31.285 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:31.459 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:31.460 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:31.467 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2026-02-21T22:18:30.192453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:34.470 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:34.646 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:34.646 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:34.654 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2026-02-21T22:18:34.192960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:34.655 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:34.830 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:34.831 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:34.838 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2026-02-21T22:18:34.192960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:34.839 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:34.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:34.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:34.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:34.840 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:35.056 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:35.056 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":819,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:35.064 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:35.065 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:35.239 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:35.240 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:35.248 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2026-02-21T22:18:34.192960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:38.250 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:38.430 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:38.430 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:38.438 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2026-02-21T22:18:38.193508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001526"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:38.440 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:38.614 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:38.614 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:38.623 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2026-02-21T22:18:38.193508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001526"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:38.623 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:38.623 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:38.623 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:38.623 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:38.623 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:38.840 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:38.840 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":823,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:38.849 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:38.849 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:39.026 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:39.026 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:39.034 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2026-02-21T22:18:38.193508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001526"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:42.037 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:42.216 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:42.216 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:42.224 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2026-02-21T22:18:42.194075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001621"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:42.226 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:42.402 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:42.402 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:42.411 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2026-02-21T22:18:42.194075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001621"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:42.412 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:42.412 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:42.412 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:42.412 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:42.412 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:42.631 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:42.631 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":827,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:42.639 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:42.639 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:42.815 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:42.816 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:42.824 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2026-02-21T22:18:42.194075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001621"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:45.826 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:46.002 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:46.002 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:46.010 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2026-02-21T22:18:44.194310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001631"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:46.010 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:46.183 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:46.184 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:46.192 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2026-02-21T22:18:44.194310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001631"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:46.193 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:46.193 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:46.194 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:46.194 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:46.194 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:46.415 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:46.415 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":831,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:46.424 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:46.425 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:46.599 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:46.600 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:46.607 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2026-02-21T22:18:46.194555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001594"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:49.609 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:49.788 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:49.789 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:49.797 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2026-02-21T22:18:48.194808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001603"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:49.799 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:49.974 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:49.975 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:49.983 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2026-02-21T22:18:48.194808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001603"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:49.984 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:49.984 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:49.984 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:49.984 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:49.985 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:50.201 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:50.201 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":835,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:50.209 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:50.210 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:50.383 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:50.384 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:50.393 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2026-02-21T22:18:50.195069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001561"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:53.394 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:53.571 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:53.571 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:53.580 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2026-02-21T22:18:52.195300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:53.581 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:53.756 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:53.757 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:53.765 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2026-02-21T22:18:52.195300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:53.766 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:53.766 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:53.767 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:53.767 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:53.767 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:53.984 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:53.984 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":838,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:53.993 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:53.993 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:54.166 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:54.166 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:54.176 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2026-02-21T22:18:52.195300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:57.178 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:57.353 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:57.354 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:57.362 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2026-02-21T22:18:56.195754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:57.363 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:57.540 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:57.540 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:57.549 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2026-02-21T22:18:56.195754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:18:57.550 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:18:57.551 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:18:57.551 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:18:57.551 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:18:57.551 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:18:57.766 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:57.766 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":842,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:18:57.775 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:18:57.775 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:18:57.949 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:18:57.949 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:18:57.958 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2026-02-21T22:18:56.195754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:00.960 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:01.138 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:01.138 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:01.145 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2026-02-21T22:19:00.196244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:01.147 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:01.322 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:01.323 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:01.330 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2026-02-21T22:19:00.196244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:01.332 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:01.332 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:01.332 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:01.332 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:01.333 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:01.553 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:01.553 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":846,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:01.562 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:01.562 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:01.738 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:01.738 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:01.746 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2026-02-21T22:19:00.196244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:04.747 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:04.925 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:04.926 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:04.934 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2026-02-21T22:19:04.196727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:04.935 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:05.107 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:05.108 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:05.116 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2026-02-21T22:19:04.196727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:05.116 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:05.117 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:05.117 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:05.117 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:05.117 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:05.334 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:05.334 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":850,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:05.343 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:05.343 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:05.517 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:05.517 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:05.526 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2026-02-21T22:19:04.196727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:08.528 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:08.706 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:08.706 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:08.714 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2026-02-21T22:19:08.197263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001509"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:08.716 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:08.893 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:08.894 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:08.902 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2026-02-21T22:19:08.197263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001509"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:08.904 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:08.904 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:08.904 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:08.904 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:08.904 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:09.126 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:09.126 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":853,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:09.136 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:09.136 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:09.313 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:09.314 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:09.322 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2026-02-21T22:19:08.197263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001509"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:12.325 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:12.504 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:12.505 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:12.514 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2026-02-21T22:19:12.197729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001485"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:12.515 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:12.695 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:12.695 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:12.704 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2026-02-21T22:19:12.197729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001485"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:12.705 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:12.705 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:12.705 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:12.705 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:12.705 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:12.925 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:12.925 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":857,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:12.934 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:12.935 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:13.109 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:13.110 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:13.118 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2026-02-21T22:19:12.197729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001485"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:15.235140+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:15:15.235140+0000","last_peered":"2026-02-21T22:15:15.235140+0000","last_clean":"2026-02-21T22:15:15.235140+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:15:15.235140+0000","last_undegraded":"2026-02-21T22:15:15.235140+0000","last_fullsized":"2026-02-21T22:15:15.235140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:16.120 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:16.299 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:16.300 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:16.308 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2026-02-21T22:19:16.198256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:16.310 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:16.485 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:16.485 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:16.493 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2026-02-21T22:19:16.198256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:16.494 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:16.495 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:16.495 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:16.495 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:16.495 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:16.712 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:16.713 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":861,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:16.722 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:16.722 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:16.899 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:16.900 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:16.908 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2026-02-21T22:19:16.198256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:19.910 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:20.089 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:20.089 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:20.097 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2026-02-21T22:19:18.198484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001519"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:20.099 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:20.276 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:20.276 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:20.285 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-21T22:19:20.198701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:20.286 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:20.286 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:20.286 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:20.287 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:20.287 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:20.503 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:20.503 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":865,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:20.511 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:20.512 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:20.687 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:20.687 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:20.695 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-21T22:19:20.198701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:23.696 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:23.874 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:23.874 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:23.883 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2026-02-21T22:19:22.198923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:23.883 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:24.057 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:24.058 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:24.066 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2026-02-21T22:19:22.198923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:24.067 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:24.067 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:24.067 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:24.068 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:24.068 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:24.285 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:24.285 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":869,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:24.294 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:24.294 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:24.472 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:24.472 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:24.480 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2026-02-21T22:19:24.199168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001438"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:27.482 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:27.660 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:27.660 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:27.669 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2026-02-21T22:19:26.199379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001386"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:27.670 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:27.847 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:27.848 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:27.856 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2026-02-21T22:19:26.199379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001386"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:27.858 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:27.858 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:27.858 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:27.858 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:27.859 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:28.074 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:28.074 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":872,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:28.084 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:28.084 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:28.260 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:28.260 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:28.269 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2026-02-21T22:19:28.199605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:31.272 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:31.449 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:31.449 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:31.458 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2026-02-21T22:19:30.199834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001350"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:31.459 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:31.636 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:31.637 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:31.645 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2026-02-21T22:19:30.199834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001350"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:31.646 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:31.646 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:31.646 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:31.646 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:31.647 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:31.862 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:31.862 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":876,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:31.871 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:31.871 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:32.046 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:32.046 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:32.054 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2026-02-21T22:19:30.199834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001350"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:35.056 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:35.234 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:35.235 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:35.243 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2026-02-21T22:19:34.200431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001507"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:35.245 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:35.421 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:35.421 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:35.429 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2026-02-21T22:19:34.200431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001507"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:35.431 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:35.431 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:35.431 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:35.431 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:35.432 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:35.653 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:35.653 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":880,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:35.661 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:35.661 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:35.837 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:35.838 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:35.846 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2026-02-21T22:19:34.200431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001507"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:38.847 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:39.024 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:39.024 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:39.031 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2026-02-21T22:19:38.201032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001653"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:39.032 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:39.208 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:39.208 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:39.216 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2026-02-21T22:19:38.201032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001653"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:39.217 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:39.218 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:39.218 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:39.218 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:39.218 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:39.435 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:39.435 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":884,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:39.444 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:39.444 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:39.620 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:39.621 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:39.629 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2026-02-21T22:19:38.201032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001653"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.236046+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:15:30.236046+0000","last_peered":"2026-02-21T22:15:30.236046+0000","last_clean":"2026-02-21T22:15:30.236046+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:15:30.236046+0000","last_undegraded":"2026-02-21T22:15:30.236046+0000","last_fullsized":"2026-02-21T22:15:30.236046+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235798+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:15:30.235798+0000","last_peered":"2026-02-21T22:15:30.235798+0000","last_clean":"2026-02-21T22:15:30.235798+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:15:30.235798+0000","last_undegraded":"2026-02-21T22:15:30.235798+0000","last_fullsized":"2026-02-21T22:15:30.235798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:30.235813+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:15:30.235813+0000","last_peered":"2026-02-21T22:15:30.235813+0000","last_clean":"2026-02-21T22:15:30.235813+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:15:30.235813+0000","last_undegraded":"2026-02-21T22:15:30.235813+0000","last_fullsized":"2026-02-21T22:15:30.235813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:42.632 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:42.809 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:42.810 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:42.818 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2026-02-21T22:19:42.201555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001720"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:42.818 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:42.995 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:42.996 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:43.005 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2026-02-21T22:19:42.201555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001720"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:43.006 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:43.006 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:43.007 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:43.007 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:35.236079+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:15:35.236079+0000', 'last_peered': '2026-02-21T22:15:35.236079+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:15:35.236079+0000', 'last_undegraded': '2026-02-21T22:15:35.236079+0000', 'last_fullsized': '2026-02-21T22:15:35.236079+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:43.007 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:43.223 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:43.224 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":888,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:43.232 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:43.232 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:43.409 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:43.409 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:43.417 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2026-02-21T22:19:42.201555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001720"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:35.236079+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:15:35.236079+0000","last_peered":"2026-02-21T22:15:35.236079+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:15:35.236079+0000","last_undegraded":"2026-02-21T22:15:35.236079+0000","last_fullsized":"2026-02-21T22:15:35.236079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:46.420 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:46.598 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:46.598 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:46.606 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2026-02-21T22:19:46.202067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001636"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:46.609 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:46.784 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:46.785 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:46.793 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2026-02-21T22:19:46.202067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001636"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:46.793 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:46.793 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:46.793 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:46.793 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:46.793 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:47.009 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:47.009 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":891,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:47.018 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:47.018 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:47.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:47.196 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:47.205 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2026-02-21T22:19:46.202067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001636"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:50.206 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:50.385 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:50.385 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:50.393 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2026-02-21T22:19:50.202539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001506"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:50.394 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:50.569 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:50.570 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:50.577 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2026-02-21T22:19:50.202539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001506"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:50.578 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:50.579 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:15:40.236871+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:15:40.236871+0000', 'last_peered': '2026-02-21T22:15:40.236871+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:15:40.236871+0000', 'last_undegraded': '2026-02-21T22:15:40.236871+0000', 'last_fullsized': '2026-02-21T22:15:40.236871+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:50.579 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:50.579 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:50.579 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:50.805 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:50.805 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":895,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:50.814 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:50.814 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:50.992 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:50.992 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:51.001 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2026-02-21T22:19:50.202539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001506"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:15:40.236871+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:15:40.236871+0000","last_peered":"2026-02-21T22:15:40.236871+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:15:40.236871+0000","last_undegraded":"2026-02-21T22:15:40.236871+0000","last_fullsized":"2026-02-21T22:15:40.236871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:54.003 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:54.187 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:54.187 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:54.196 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2026-02-21T22:19:52.202737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:54.197 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:54.379 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:54.379 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:54.388 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2026-02-21T22:19:54.202957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:54.389 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:54.389 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:54.390 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:54.390 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:54.390 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:54.610 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:54.610 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":899,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:54.620 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:54.620 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:54.798 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:54.799 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:54.807 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2026-02-21T22:19:54.202957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:57.808 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:57.984 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:57.984 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:57.993 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2026-02-21T22:19:56.203167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001359"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:57.993 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:58.169 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:58.169 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:58.177 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2026-02-21T22:19:56.203167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001359"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:19:58.177 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:19:58.177 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:19:58.178 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:19:58.178 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:19:58.178 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:19:58.392 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:58.392 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":903,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:19:58.400 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:19:58.401 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:19:58.573 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:19:58.574 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:19:58.581 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2026-02-21T22:19:58.203394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001326"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:01.583 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:01.759 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:01.760 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:01.767 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2026-02-21T22:20:00.203579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:01.768 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:01.943 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:01.944 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:01.951 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2026-02-21T22:20:00.203579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:01.951 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:01.952 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:01.952 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:01.952 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:01.952 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:02.171 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:02.172 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":906,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:02.181 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:02.181 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:02.356 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:02.357 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:02.366 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2026-02-21T22:20:02.203790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001250"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:05.368 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:05.547 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:05.548 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:05.556 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2026-02-21T22:20:04.204000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001262"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:05.556 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:05.732 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:05.732 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:05.741 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2026-02-21T22:20:04.204000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001262"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:05.742 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:05.742 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:05.743 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:05.743 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:05.743 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":910,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:05.969 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:05.970 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:06.148 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:06.148 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2026-02-21T22:20:04.204000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001262"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:09.158 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:09.336 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:09.336 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:09.344 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2026-02-21T22:20:08.204485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001317"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:09.346 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:09.522 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:09.522 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:09.530 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2026-02-21T22:20:08.204485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001317"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:09.532 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:09.532 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:09.532 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:09.532 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:09.533 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:09.750 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:09.750 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":914,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:09.759 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:09.759 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:09.934 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:09.935 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:09.942 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2026-02-21T22:20:08.204485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001317"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:12.945 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:13.120 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:13.121 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:13.129 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2026-02-21T22:20:12.204912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:13.130 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:13.308 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:13.308 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:13.318 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2026-02-21T22:20:12.204912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:13.319 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:13.320 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:13.320 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:13.320 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:13.320 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:13.536 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:13.536 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":918,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:13.545 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:13.545 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:13.723 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:13.724 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:13.732 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2026-02-21T22:20:12.204912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:15:55.237681+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:15:55.237681+0000","last_peered":"2026-02-21T22:15:55.237681+0000","last_clean":"2026-02-21T22:15:55.237681+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:15:55.237681+0000","last_undegraded":"2026-02-21T22:15:55.237681+0000","last_fullsized":"2026-02-21T22:15:55.237681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:16.735 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:16.909 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:16.910 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:16.918 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2026-02-21T22:20:16.205374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:16.919 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:17.094 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:17.094 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:17.103 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2026-02-21T22:20:16.205374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:17.104 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:17.104 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:17.104 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:17.104 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:17.104 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:17.321 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:17.321 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":922,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:17.329 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:17.330 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:17.507 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:17.507 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:17.516 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2026-02-21T22:20:16.205374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:20.518 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:20.692 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:20.693 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:20.701 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2026-02-21T22:20:20.205879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:20.702 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:20.876 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:20.877 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:20.885 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2026-02-21T22:20:20.205879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:20.886 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:20.887 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:20.887 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:20.887 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:20.887 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:21.103 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:21.103 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":925,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:21.112 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:21.112 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:21.286 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:21.287 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:21.295 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2026-02-21T22:20:20.205879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:16:00.238261+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:16:00.238261+0000","last_peered":"2026-02-21T22:16:00.238261+0000","last_clean":"2026-02-21T22:16:00.238261+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:16:00.238261+0000","last_undegraded":"2026-02-21T22:16:00.238261+0000","last_fullsized":"2026-02-21T22:16:00.238261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:24.296 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:24.470 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:24.470 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:24.479 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2026-02-21T22:20:24.206345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:24.479 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:24.658 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:24.659 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:24.667 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2026-02-21T22:20:24.206345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:24.668 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:24.669 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:24.669 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:24.669 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:24.669 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:24.887 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:24.887 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":929,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:24.896 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:24.896 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:25.073 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:25.073 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:25.081 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2026-02-21T22:20:24.206345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:28.083 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:28.262 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:28.263 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:28.271 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2026-02-21T22:20:28.206850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:28.273 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:28.453 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:28.453 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:28.462 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2026-02-21T22:20:28.206850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:28.462 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:28.462 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:28.463 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:28.463 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:28.463 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:28.678 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:28.678 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":933,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:28.688 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:28.688 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:28.867 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:28.867 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:28.875 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2026-02-21T22:20:28.206850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:31.877 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:32.052 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:32.053 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:32.060 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2026-02-21T22:20:30.207062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001453"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:32.062 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:32.234 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:32.235 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:32.243 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2026-02-21T22:20:32.207314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:32.243 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:32.243 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:32.244 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:32.244 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:32.244 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:32.462 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:32.462 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":937,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:32.472 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:32.472 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:32.651 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:32.651 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:32.659 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2026-02-21T22:20:32.207314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:35.661 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:35.835 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:35.836 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:35.843 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2026-02-21T22:20:34.207537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001465"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:35.844 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:36.019 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:36.020 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:36.028 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2026-02-21T22:20:34.207537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001465"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:36.029 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:36.029 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:36.030 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:36.030 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:36.030 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:36.253 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:36.253 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":941,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:36.262 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:36.262 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:36.437 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:36.438 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:36.446 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2026-02-21T22:20:36.207779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:39.448 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:39.627 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:39.628 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:39.636 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2026-02-21T22:20:38.207998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001427"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:39.637 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:39.815 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:39.815 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:39.824 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2026-02-21T22:20:38.207998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001427"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:39.826 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:39.826 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:39.827 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:39.827 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:39.827 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:40.043 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:40.043 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":944,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:40.051 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:40.051 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:40.222 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:40.223 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:40.231 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2026-02-21T22:20:40.208252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:43.232 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:43.409 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:43.410 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:43.418 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2026-02-21T22:20:42.208465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:43.419 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:43.594 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:43.594 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:43.602 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2026-02-21T22:20:42.208465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:43.602 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:43.602 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:43.602 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:43.602 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:43.603 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:43.817 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:43.817 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":948,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:43.826 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:43.826 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:44.002 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:44.003 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:44.011 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2026-02-21T22:20:42.208465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:47.014 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:47.194 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:47.194 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:47.203 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2026-02-21T22:20:46.208944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:47.205 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:47.381 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:47.382 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:47.391 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2026-02-21T22:20:46.208944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:47.392 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:47.392 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:47.392 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:47.393 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:47.393 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:47.607 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:47.607 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":952,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:47.615 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:47.616 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:47.795 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:47.795 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:47.804 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2026-02-21T22:20:46.208944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:50.806 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:50.983 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:50.983 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:50.991 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2026-02-21T22:20:50.209456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:50.992 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:51.166 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:51.167 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:51.175 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2026-02-21T22:20:50.209456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:51.175 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:51.175 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:51.176 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:51.176 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:51.176 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:51.401 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:51.401 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":956,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:51.412 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:51.412 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:51.587 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:51.587 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:51.596 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2026-02-21T22:20:50.209456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:54.597 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:54.773 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:54.773 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:54.781 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2026-02-21T22:20:54.209901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:54.782 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:54.956 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:54.956 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:54.964 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2026-02-21T22:20:54.209901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:54.965 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:54.965 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:54.965 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:54.965 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:54.965 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:55.182 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:55.182 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":960,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T22:06:02.103227+0000","services":{}},"progress_events":{}} 2026-02-21T22:20:55.191 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:55.191 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:55.368 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:55.368 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:55.378 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2026-02-21T22:20:54.209901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:58.379 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:58.554 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:58.554 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:58.562 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2026-02-21T22:20:58.210342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001398"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:58.562 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:58.736 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:58.736 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:58.744 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2026-02-21T22:20:58.210342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001398"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:20:58.744 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:20:58.744 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:20:58.744 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:20:58.745 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:20:58.745 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:20:58.963 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:58.964 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":963,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":4,"modified":"2026-02-21T22:20:58.210371+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-21T22:20:58.972 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:20:58.973 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:20:59.143 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:20:59.144 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:20:59.152 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2026-02-21T22:20:58.210342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001398"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:02.154 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:02.332 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:02.333 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:02.342 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2026-02-21T22:21:02.210826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001370"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:02.343 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:02.520 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:02.521 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:02.529 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2026-02-21T22:21:02.210826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001370"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:02.530 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:02.531 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:02.531 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:02.531 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:02.531 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:02.750 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:02.750 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":967,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:02.759 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:02.760 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:02.935 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:02.935 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:02.944 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2026-02-21T22:21:02.210826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001370"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:05.945 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:06.125 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:06.126 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:06.135 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2026-02-21T22:21:04.211050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:06.136 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:06.315 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:06.315 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:06.324 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2026-02-21T22:21:06.211333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:06.326 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:06.326 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:06.326 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:06.326 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:06.327 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:06.551 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:06.551 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":971,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:06.560 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:06.561 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:06.736 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:06.737 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:06.745 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2026-02-21T22:21:06.211333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:09.746 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:09.923 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:09.924 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:09.931 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2026-02-21T22:21:08.211577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:09.933 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:10.110 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:10.110 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:10.118 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2026-02-21T22:21:08.211577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:10.119 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:10.119 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:10.119 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:10.119 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:10.119 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:10.336 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:10.336 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":975,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:10.346 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:10.346 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:10.521 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:10.521 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:10.529 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2026-02-21T22:21:10.211840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001497"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:13.531 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:13.709 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:13.709 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:13.717 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2026-02-21T22:21:12.212044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:13.719 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:13.896 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:13.897 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:13.905 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2026-02-21T22:21:12.212044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:13.906 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:13.906 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:13.907 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:13.907 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:13.907 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:14.120 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:14.120 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":978,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:14.129 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:14.129 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:14.303 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:14.303 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:14.312 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2026-02-21T22:21:14.212314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:17.314 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:17.490 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:17.490 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:17.499 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2026-02-21T22:21:16.212519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:17.500 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:17.675 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:17.675 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:17.683 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2026-02-21T22:21:16.212519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:17.684 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:17.684 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:17.685 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:17.685 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:17.685 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:17.905 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:17.906 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":982,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:17.915 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:17.915 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:18.091 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:18.092 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:18.100 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2026-02-21T22:21:16.212519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:15.243736+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:17:15.243736+0000","last_peered":"2026-02-21T22:17:15.243736+0000","last_clean":"2026-02-21T22:17:15.243736+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:17:15.243736+0000","last_undegraded":"2026-02-21T22:17:15.243736+0000","last_fullsized":"2026-02-21T22:17:15.243736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:21.103 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:21.283 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:21.283 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:21.292 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2026-02-21T22:21:20.213007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:21.293 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:21.469 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:21.469 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:21.478 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2026-02-21T22:21:20.213007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:21.479 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:21.479 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:21.480 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:21.480 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:21.480 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:21.695 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:21.695 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":986,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:21.704 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:21.704 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:21.880 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:21.881 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:21.888 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2026-02-21T22:21:20.213007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:24.890 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:25.068 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:25.069 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:25.077 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2026-02-21T22:21:24.213467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:25.078 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:25.255 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:25.255 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:25.263 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2026-02-21T22:21:24.213467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:25.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:25.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:25.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:25.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:25.266 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:25.485 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:25.485 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":990,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:25.494 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:25.494 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:25.670 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:25.671 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:25.679 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2026-02-21T22:21:24.213467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:28.681 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:28.864 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:28.865 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:28.873 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2026-02-21T22:21:28.213954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:28.874 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:29.048 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:29.048 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:29.056 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2026-02-21T22:21:28.213954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:29.058 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:29.058 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:29.058 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:29.058 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:29.058 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:29.274 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:29.275 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":994,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:29.284 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:29.284 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:29.461 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:29.462 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:29.470 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2026-02-21T22:21:28.213954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001434"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:32.472 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:32.647 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:32.648 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:32.655 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2026-02-21T22:21:32.214495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:32.655 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:32.832 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:32.832 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:32.841 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2026-02-21T22:21:32.214495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:32.842 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:32.842 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:32.843 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:32.843 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:32.843 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:33.060 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:33.060 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":997,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:33.069 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:33.069 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:33.243 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:33.243 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:33.250 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2026-02-21T22:21:32.214495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:36.251 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:36.431 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:36.431 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:36.440 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2026-02-21T22:21:36.215040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001573"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:36.440 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:36.616 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2026-02-21T22:21:36.215040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001573"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:36.624 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:36.624 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:36.624 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:36.624 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:36.624 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:36.842 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:36.843 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1001,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:36.851 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:36.851 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:37.026 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:37.026 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:37.034 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2026-02-21T22:21:36.215040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001573"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:40.036 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:40.215 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:40.215 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:40.224 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2026-02-21T22:21:38.215262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001540"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:40.224 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:40.399 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:40.400 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:40.409 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2026-02-21T22:21:40.215502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001547"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:40.410 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:40.410 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:40.411 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:40.411 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:40.411 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:40.629 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:40.630 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1005,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:40.638 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:40.638 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:40.819 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:40.820 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:40.828 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2026-02-21T22:21:40.215502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001547"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:43.831 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:44.008 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:44.008 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:44.016 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2026-02-21T22:21:42.215719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001490"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:44.018 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:44.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:44.195 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:44.203 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2026-02-21T22:21:42.215719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001490"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:44.204 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:44.204 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:44.204 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:44.205 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:44.205 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:44.424 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:44.424 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1009,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:44.434 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:44.434 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:44.613 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:44.613 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:44.621 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":517,"stamp":"2026-02-21T22:21:44.216016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001520"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244857+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:17:35.244857+0000","last_peered":"2026-02-21T22:17:35.244857+0000","last_clean":"2026-02-21T22:17:35.244857+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:17:35.244857+0000","last_undegraded":"2026-02-21T22:17:35.244857+0000","last_fullsized":"2026-02-21T22:17:35.244857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244874+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:17:35.244874+0000","last_peered":"2026-02-21T22:17:35.244874+0000","last_clean":"2026-02-21T22:17:35.244874+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:17:35.244874+0000","last_undegraded":"2026-02-21T22:17:35.244874+0000","last_fullsized":"2026-02-21T22:17:35.244874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:47.623 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:47.800 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:47.800 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:47.809 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2026-02-21T22:21:46.216238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:47.810 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:47.985 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:47.985 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:47.993 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2026-02-21T22:21:46.216238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:47.995 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:47.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:47.995 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:47.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:47.995 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:48.215 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:48.215 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1013,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:48.225 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:48.225 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:48.400 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:48.400 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:48.408 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2026-02-21T22:21:48.216456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:17:35.244942+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:17:35.244942+0000","last_peered":"2026-02-21T22:17:35.244942+0000","last_clean":"2026-02-21T22:17:35.244942+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:17:35.244942+0000","last_undegraded":"2026-02-21T22:17:35.244942+0000","last_fullsized":"2026-02-21T22:17:35.244942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:51.411 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:51.586 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:51.586 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:51.594 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2026-02-21T22:21:50.216676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:51.596 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:51.770 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:51.771 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:51.779 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2026-02-21T22:21:50.216676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:51.780 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:51.780 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:51.781 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:51.781 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:40.245168+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:17:40.245168+0000', 'last_peered': '2026-02-21T22:17:40.245168+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:17:40.245168+0000', 'last_undegraded': '2026-02-21T22:17:40.245168+0000', 'last_fullsized': '2026-02-21T22:17:40.245168+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:51.781 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:51.998 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:51.998 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1016,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:52.007 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:52.008 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:52.181 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:52.182 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:52.189 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2026-02-21T22:21:50.216676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001413"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:40.245168+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:17:40.245168+0000","last_peered":"2026-02-21T22:17:40.245168+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:17:40.245168+0000","last_undegraded":"2026-02-21T22:17:40.245168+0000","last_fullsized":"2026-02-21T22:17:40.245168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:55.191 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:55.367 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:55.367 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:55.376 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-21T22:21:54.217200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001481"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:19:45.254520+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:19:45.254520+0000","last_peered":"2026-02-21T22:19:45.254520+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:19:45.254520+0000","last_undegraded":"2026-02-21T22:19:45.254520+0000","last_fullsized":"2026-02-21T22:19:45.254520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:55.377 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:55.557 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:55.558 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:55.567 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-21T22:21:54.217200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001481"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:19:45.254520+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:19:45.254520+0000","last_peered":"2026-02-21T22:19:45.254520+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:19:45.254520+0000","last_undegraded":"2026-02-21T22:19:45.254520+0000","last_fullsized":"2026-02-21T22:19:45.254520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:55.569 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:21:55.569 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:17:45.245518+0000', 'last_change': '2026-02-21T22:07:20.200622+0000', 'last_active': '2026-02-21T22:17:45.245518+0000', 'last_peered': '2026-02-21T22:17:45.245518+0000', 'last_clean': '2026-02-21T22:05:04.195147+0000', 'last_became_active': '2026-02-21T22:05:13.221015+0000', 'last_became_peered': '2026-02-21T22:05:13.221015+0000', 'last_unstale': '2026-02-21T22:17:45.245518+0000', 'last_undegraded': '2026-02-21T22:17:45.245518+0000', 'last_fullsized': '2026-02-21T22:17:45.245518+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:21:55.569 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:21:55.569 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-21T22:19:45.254520+0000', 'last_change': '2026-02-21T22:07:15.200569+0000', 'last_active': '2026-02-21T22:19:45.254520+0000', 'last_peered': '2026-02-21T22:19:45.254520+0000', 'last_clean': '2026-02-21T22:05:09.972420+0000', 'last_became_active': '2026-02-21T22:05:13.219519+0000', 'last_became_peered': '2026-02-21T22:05:13.219519+0000', 'last_unstale': '2026-02-21T22:19:45.254520+0000', 'last_undegraded': '2026-02-21T22:19:45.254520+0000', 'last_fullsized': '2026-02-21T22:19:45.254520+0000', 'mapping_epoch': 16, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:21:55.570 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:21:55.789 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:55.789 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1020,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771711512,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:21:55.797 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:21:55.798 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:55.972 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:55.973 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:55.981 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-21T22:21:54.217200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001481"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:17:45.245518+0000","last_change":"2026-02-21T22:07:20.200622+0000","last_active":"2026-02-21T22:17:45.245518+0000","last_peered":"2026-02-21T22:17:45.245518+0000","last_clean":"2026-02-21T22:05:04.195147+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:17:45.245518+0000","last_undegraded":"2026-02-21T22:17:45.245518+0000","last_fullsized":"2026-02-21T22:17:45.245518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-21T22:19:45.254520+0000","last_change":"2026-02-21T22:07:15.200569+0000","last_active":"2026-02-21T22:19:45.254520+0000","last_peered":"2026-02-21T22:19:45.254520+0000","last_clean":"2026-02-21T22:05:09.972420+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:19:45.254520+0000","last_undegraded":"2026-02-21T22:19:45.254520+0000","last_fullsized":"2026-02-21T22:19:45.254520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:58.984 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:21:59.159 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:21:59.160 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:21:59.169 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2026-02-21T22:21:58.217693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54216,"kb_used_data":296,"kb_used_omap":16,"kb_used_meta":53871,"kb_avail":188689464,"statfs":{"total":193273528320,"available":193218011136,"internally_reserved":0,"allocated":303104,"data_stored":89534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16402,"internal_metadata":55164910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":-1,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":-3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001454"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:10.247428+0000","last_change":"2026-02-21T22:05:13.220768+0000","last_active":"2026-02-21T22:18:10.247428+0000","last_peered":"2026-02-21T22:18:10.247428+0000","last_clean":"2026-02-21T22:18:10.247428+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:18:10.247428+0000","last_undegraded":"2026-02-21T22:18:10.247428+0000","last_fullsized":"2026-02-21T22:18:10.247428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:21:31.901015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:18:05.246961+0000","last_change":"2026-02-21T22:05:13.220436+0000","last_active":"2026-02-21T22:18:05.246961+0000","last_peered":"2026-02-21T22:18:05.246961+0000","last_clean":"2026-02-21T22:18:05.246961+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:18:05.246961+0000","last_undegraded":"2026-02-21T22:18:05.246961+0000","last_fullsized":"2026-02-21T22:18:05.246961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:28:24.464224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254387+0000","last_change":"2026-02-21T22:05:07.330259+0000","last_active":"2026-02-21T22:19:40.254387+0000","last_peered":"2026-02-21T22:19:40.254387+0000","last_clean":"2026-02-21T22:19:40.254387+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:19:40.254387+0000","last_undegraded":"2026-02-21T22:19:40.254387+0000","last_fullsized":"2026-02-21T22:19:40.254387+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:31.670047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254009+0000","last_change":"2026-02-21T22:05:13.220927+0000","last_active":"2026-02-21T22:19:40.254009+0000","last_peered":"2026-02-21T22:19:40.254009+0000","last_clean":"2026-02-21T22:19:40.254009+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:19:40.254009+0000","last_undegraded":"2026-02-21T22:19:40.254009+0000","last_fullsized":"2026-02-21T22:19:40.254009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:34:35.991472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:21:52.338396+0000","last_change":"2026-02-21T22:21:52.338145+0000","last_active":"2026-02-21T22:21:52.338396+0000","last_peered":"2026-02-21T22:21:52.338396+0000","last_clean":"2026-02-21T22:21:52.338396+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:21:52.338396+0000","last_undegraded":"2026-02-21T22:21:52.338396+0000","last_fullsized":"2026-02-21T22:21:52.338396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:45:57.743202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":25,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:21:52.256535+0000","last_change":"2026-02-21T22:21:52.256473+0000","last_active":"2026-02-21T22:21:52.256535+0000","last_peered":"2026-02-21T22:21:52.256535+0000","last_clean":"2026-02-21T22:21:52.256535+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:21:52.256535+0000","last_undegraded":"2026-02-21T22:21:52.256535+0000","last_fullsized":"2026-02-21T22:21:52.256535+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:53:38.364495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:40.254032+0000","last_change":"2026-02-21T22:05:13.221071+0000","last_active":"2026-02-21T22:19:40.254032+0000","last_peered":"2026-02-21T22:19:40.254032+0000","last_clean":"2026-02-21T22:19:40.254032+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:19:40.254032+0000","last_undegraded":"2026-02-21T22:19:40.254032+0000","last_fullsized":"2026-02-21T22:19:40.254032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:49:12.546060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T22:19:15.252265+0000","last_change":"2026-02-21T22:05:13.220739+0000","last_active":"2026-02-21T22:19:15.252265+0000","last_peered":"2026-02-21T22:19:15.252265+0000","last_clean":"2026-02-21T22:19:15.252265+0000","last_became_active":"2026-02-21T22:05:13.220445+0000","last_became_peered":"2026-02-21T22:05:13.220445+0000","last_unstale":"2026-02-21T22:19:15.252265+0000","last_undegraded":"2026-02-21T22:19:15.252265+0000","last_fullsized":"2026-02-21T22:19:15.252265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:46:38.561332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6900,"internal_metadata":27583756},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705873,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:21:59.171 INFO:tasks.osd_recovery.ceph_manager:clean! 2026-02-21T22:21:59.171 INFO:tasks.osd_recovery:rados -p rbd bench 20 write -b 1 --no-cleanup 2026-02-21T22:21:59.172 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd bench 20 write -b 1 --no-cleanup 2026-02-21T22:21:59.208 INFO:teuthology.orchestra.run.trial060.stdout:hints = 1 2026-02-21T22:21:59.208 INFO:teuthology.orchestra.run.trial060.stdout:Maintaining 16 concurrent writes of 1 bytes to objects of size 1 for up to 20 seconds or 0 objects 2026-02-21T22:21:59.208 INFO:teuthology.orchestra.run.trial060.stdout:Object prefix: benchmark_data_trial060_102781 2026-02-21T22:21:59.208 INFO:teuthology.orchestra.run.trial060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T22:21:59.208 INFO:teuthology.orchestra.run.trial060.stdout: 0 0 0 0 0 0 - 0 2026-02-21T22:22:00.208 INFO:teuthology.orchestra.run.trial060.stdout: 1 16 7637 7621 0.00726739 0.00726795 0.00290943 0.00209475 2026-02-21T22:22:01.208 INFO:teuthology.orchestra.run.trial060.stdout: 2 16 13772 13756 0.00655889 0.00585079 0.00211996 0.00232404 2026-02-21T22:22:02.208 INFO:teuthology.orchestra.run.trial060.stdout: 3 16 18745 18729 0.00595336 0.00474262 0.00333499 0.00256085 2026-02-21T22:22:03.208 INFO:teuthology.orchestra.run.trial060.stdout: 4 16 22926 22910 0.00546178 0.00398731 0.00176232 0.00279096 2026-02-21T22:22:04.208 INFO:teuthology.orchestra.run.trial060.stdout: 5 16 26623 26607 0.00507452 0.00352573 0.00691141 0.00300451 2026-02-21T22:22:05.208 INFO:teuthology.orchestra.run.trial060.stdout: 6 16 29873 29857 0.00474531 0.00309944 0.00238037 0.00321293 2026-02-21T22:22:06.208 INFO:teuthology.orchestra.run.trial060.stdout: 7 16 32815 32799 0.00446819 0.00280571 0.00640813 0.00341231 2026-02-21T22:22:07.208 INFO:teuthology.orchestra.run.trial060.stdout: 8 16 35485 35469 0.00422793 0.00254631 0.00647436 0.00360673 2026-02-21T22:22:08.208 INFO:teuthology.orchestra.run.trial060.stdout: 9 16 38082 38066 0.00403333 0.00247669 0.00311676 0.0037806 2026-02-21T22:22:09.208 INFO:teuthology.orchestra.run.trial060.stdout: 10 16 40419 40403 0.00385286 0.00222874 0.00859747 0.00395805 2026-02-21T22:22:10.208 INFO:teuthology.orchestra.run.trial060.stdout: 11 16 42517 42501 0.00368448 0.00200081 0.00827766 0.00413894 2026-02-21T22:22:11.208 INFO:teuthology.orchestra.run.trial060.stdout: 12 16 44503 44487 0.00353526 0.001894 0.0045558 0.00431344 2026-02-21T22:22:12.208 INFO:teuthology.orchestra.run.trial060.stdout: 13 16 46440 46424 0.0034054 0.00184727 0.00430084 0.00447756 2026-02-21T22:22:13.209 INFO:teuthology.orchestra.run.trial060.stdout: 14 16 48290 48274 0.00328817 0.0017643 0.0115724 0.00463738 2026-02-21T22:22:14.208 INFO:teuthology.orchestra.run.trial060.stdout: 15 16 50012 49996 0.00317844 0.00164223 0.00887583 0.00479822 2026-02-21T22:22:15.209 INFO:teuthology.orchestra.run.trial060.stdout: 16 16 51698 51682 0.00308027 0.00160789 0.0129731 0.00495049 2026-02-21T22:22:16.209 INFO:teuthology.orchestra.run.trial060.stdout: 17 16 53335 53319 0.0029909 0.00156116 0.00960414 0.00509937 2026-02-21T22:22:17.209 INFO:teuthology.orchestra.run.trial060.stdout: 18 16 54868 54852 0.00290596 0.00146198 0.00432802 0.00524888 2026-02-21T22:22:18.209 INFO:teuthology.orchestra.run.trial060.stdout: 19 16 56315 56299 0.00282564 0.00137997 0.0223548 0.00539667 2026-02-21T22:22:19.209 INFO:teuthology.orchestra.run.trial060.stdout:2026-02-21T22:22:19.209034+0000 min lat: 0.000507796 max lat: 0.0343651 avg lat: 0.00554554 2026-02-21T22:22:19.209 INFO:teuthology.orchestra.run.trial060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T22:22:19.209 INFO:teuthology.orchestra.run.trial060.stdout: 20 14 57693 57679 0.00275015 0.00131607 0.0207203 0.00554554 2026-02-21T22:22:20.209 INFO:teuthology.orchestra.run.trial060.stdout:Total time run: 20.007 2026-02-21T22:22:20.209 INFO:teuthology.orchestra.run.trial060.stdout:Total writes made: 57693 2026-02-21T22:22:20.209 INFO:teuthology.orchestra.run.trial060.stdout:Write size: 1 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Object size: 1 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Bandwidth (MB/sec): 0.00275005 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Stddev Bandwidth: 0.00161018 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Max bandwidth (MB/sec): 0.00726795 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Min bandwidth (MB/sec): 0.00131607 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Average IOPS: 2883 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Stddev IOPS: 1688.39 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Max IOPS: 7621 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Min IOPS: 1380 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Average Latency(s): 0.00554684 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Stddev Latency(s): 0.00408682 2026-02-21T22:22:20.210 INFO:teuthology.orchestra.run.trial060.stdout:Max latency(s): 0.0343651 2026-02-21T22:22:20.211 INFO:teuthology.orchestra.run.trial060.stdout:Min latency(s): 0.000507796 2026-02-21T22:22:20.217 INFO:tasks.osd_recovery:rados -p rbd put foo.1 /etc/passwd 2026-02-21T22:22:20.217 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.1 /etc/passwd 2026-02-21T22:22:20.297 INFO:tasks.osd_recovery:rados -p rbd put foo.2 /etc/passwd 2026-02-21T22:22:20.297 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.2 /etc/passwd 2026-02-21T22:22:20.376 INFO:tasks.osd_recovery:rados -p rbd put foo.3 /etc/passwd 2026-02-21T22:22:20.376 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.3 /etc/passwd 2026-02-21T22:22:20.454 INFO:tasks.osd_recovery:rados -p rbd put foo.4 /etc/passwd 2026-02-21T22:22:20.454 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.4 /etc/passwd 2026-02-21T22:22:20.531 INFO:tasks.osd_recovery:rados -p rbd put foo.5 /etc/passwd 2026-02-21T22:22:20.532 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.5 /etc/passwd 2026-02-21T22:22:20.611 INFO:tasks.osd_recovery:rados -p rbd put foo.6 /etc/passwd 2026-02-21T22:22:20.612 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.6 /etc/passwd 2026-02-21T22:22:20.691 INFO:tasks.osd_recovery:rados -p rbd put foo.7 /etc/passwd 2026-02-21T22:22:20.692 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.7 /etc/passwd 2026-02-21T22:22:20.770 INFO:tasks.osd_recovery:rados -p rbd put foo.8 /etc/passwd 2026-02-21T22:22:20.771 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.8 /etc/passwd 2026-02-21T22:22:20.851 INFO:tasks.osd_recovery:rados -p rbd put foo.9 /etc/passwd 2026-02-21T22:22:20.851 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.9 /etc/passwd 2026-02-21T22:22:20.931 INFO:tasks.osd_recovery:rados -p rbd put foo.10 /etc/passwd 2026-02-21T22:22:20.931 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.10 /etc/passwd 2026-02-21T22:22:21.010 INFO:tasks.osd_recovery:rados -p rbd put foo.11 /etc/passwd 2026-02-21T22:22:21.010 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.11 /etc/passwd 2026-02-21T22:22:21.091 INFO:tasks.osd_recovery:rados -p rbd put foo.12 /etc/passwd 2026-02-21T22:22:21.092 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.12 /etc/passwd 2026-02-21T22:22:21.170 INFO:tasks.osd_recovery:rados -p rbd put foo.13 /etc/passwd 2026-02-21T22:22:21.171 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.13 /etc/passwd 2026-02-21T22:22:21.250 INFO:tasks.osd_recovery:rados -p rbd put foo.14 /etc/passwd 2026-02-21T22:22:21.250 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.14 /etc/passwd 2026-02-21T22:22:21.328 INFO:tasks.osd_recovery:rados -p rbd put foo.15 /etc/passwd 2026-02-21T22:22:21.329 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.15 /etc/passwd 2026-02-21T22:22:21.408 INFO:tasks.osd_recovery:rados -p rbd put foo.16 /etc/passwd 2026-02-21T22:22:21.408 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.16 /etc/passwd 2026-02-21T22:22:21.487 INFO:tasks.osd_recovery:rados -p rbd put foo.17 /etc/passwd 2026-02-21T22:22:21.488 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.17 /etc/passwd 2026-02-21T22:22:21.566 INFO:tasks.osd_recovery:rados -p rbd put foo.18 /etc/passwd 2026-02-21T22:22:21.566 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.18 /etc/passwd 2026-02-21T22:22:21.643 INFO:tasks.osd_recovery:rados -p rbd put foo.19 /etc/passwd 2026-02-21T22:22:21.643 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.19 /etc/passwd 2026-02-21T22:22:21.722 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 1 2026-02-21T22:22:21.945 INFO:tasks.ceph.mon.a.trial060.stderr:2026-02-21T22:22:21.944+0000 7fb3dbff1640 -1 mon.a@0(leader).osd e17 definitely_dead 0 2026-02-21T22:22:22.350 INFO:teuthology.orchestra.run.trial060.stderr:marked in osd.0. marked in osd.1. 2026-02-21T22:22:22.362 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 3 2026-02-21T22:22:22.543 INFO:tasks.ceph.mon.a.trial060.stderr:2026-02-21T22:22:22.542+0000 7fb3dbff1640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2026-02-21T22:22:23.352 INFO:teuthology.orchestra.run.trial060.stderr:marked out osd.2. marked out osd.3. 2026-02-21T22:22:33.364 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T22:22:33.365 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T22:22:33.365 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T22:22:33.366 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T22:22:33.454 INFO:teuthology.orchestra.run.trial060.stdout:38654705880 2026-02-21T22:22:33.454 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:22:33.455 INFO:teuthology.orchestra.run.trial060.stdout:38654705881 2026-02-21T22:22:33.455 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:22:33.456 INFO:teuthology.orchestra.run.trial060.stdout:38654705880 2026-02-21T22:22:33.456 DEBUG:teuthology.parallel:result is None 2026-02-21T22:22:33.458 INFO:teuthology.orchestra.run.trial060.stdout:38654705881 2026-02-21T22:22:33.458 DEBUG:teuthology.parallel:result is None 2026-02-21T22:22:33.646 INFO:teuthology.orchestra.run.trial060.stdout:38654705879 2026-02-21T22:22:33.647 INFO:teuthology.orchestra.run.trial060.stdout:38654705880 2026-02-21T22:22:33.657 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705881 got 38654705880 for osd.1 2026-02-21T22:22:33.658 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705880 got 38654705879 for osd.0 2026-02-21T22:22:34.658 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:22:34.659 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:22:34.841 INFO:teuthology.orchestra.run.trial060.stdout:38654705881 2026-02-21T22:22:34.841 INFO:teuthology.orchestra.run.trial060.stdout:38654705880 2026-02-21T22:22:34.851 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705880 got 38654705880 for osd.0 2026-02-21T22:22:34.852 DEBUG:teuthology.parallel:result is None 2026-02-21T22:22:34.852 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705881 got 38654705881 for osd.1 2026-02-21T22:22:34.852 DEBUG:teuthology.parallel:result is None 2026-02-21T22:22:44.853 INFO:tasks.osd_recovery.ceph_manager:waiting for peering to complete 2026-02-21T22:22:44.853 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:45.043 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:45.043 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:45.054 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-21T22:22:44.223192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162560,"kb_used_data":352,"kb_used_omap":20,"kb_used_meta":162155,"kb_avail":188581120,"statfs":{"total":193273528320,"available":193107066880,"internally_reserved":0,"allocated":360448,"data_stored":117480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20981,"internal_metadata":166047243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5844,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7287,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.555546+0000","last_change":"2026-02-21T22:22:25.556143+0000","last_active":"2026-02-21T22:22:25.555546+0000","last_peered":"2026-02-21T22:22:25.555546+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:25.552595+0000","last_became_peered":"2026-02-21T22:22:25.552595+0000","last_unstale":"2026-02-21T22:22:25.555546+0000","last_undegraded":"2026-02-21T22:22:25.506596+0000","last_fullsized":"2026-02-21T22:22:25.431530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356953+0000","last_change":"2026-02-21T22:22:24.702184+0000","last_active":"2026-02-21T22:22:25.356953+0000","last_peered":"2026-02-21T22:22:25.356953+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:22:24.697010+0000","last_became_peered":"2026-02-21T22:22:24.697010+0000","last_unstale":"2026-02-21T22:22:25.356953+0000","last_undegraded":"2026-02-21T22:22:24.474837+0000","last_fullsized":"2026-02-21T22:22:24.356051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7306,"num_objects_misplaced":7306,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7306}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7177,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.537321+0000","last_change":"2026-02-21T22:22:25.551643+0000","last_active":"2026-02-21T22:22:25.537321+0000","last_peered":"2026-02-21T22:22:25.537321+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:25.534289+0000","last_became_peered":"2026-02-21T22:22:25.534289+0000","last_unstale":"2026-02-21T22:22:25.537321+0000","last_undegraded":"2026-02-21T22:22:25.430811+0000","last_fullsized":"2026-02-21T22:22:25.357439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7313,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356967+0000","last_change":"2026-02-21T22:22:24.733636+0000","last_active":"2026-02-21T22:22:25.356967+0000","last_peered":"2026-02-21T22:22:25.356967+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:22:24.729840+0000","last_became_peered":"2026-02-21T22:22:24.729840+0000","last_unstale":"2026-02-21T22:22:25.356967+0000","last_undegraded":"2026-02-21T22:22:24.558211+0000","last_fullsized":"2026-02-21T22:22:24.477081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7258,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.510735+0000","last_change":"2026-02-21T22:22:24.702122+0000","last_active":"2026-02-21T22:22:25.510735+0000","last_peered":"2026-02-21T22:22:25.510735+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:24.697165+0000","last_became_peered":"2026-02-21T22:22:24.697165+0000","last_unstale":"2026-02-21T22:22:25.510735+0000","last_undegraded":"2026-02-21T22:22:24.528989+0000","last_fullsized":"2026-02-21T22:22:24.446698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7262,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.507289+0000","last_change":"2026-02-21T22:22:24.701417+0000","last_active":"2026-02-21T22:22:25.507289+0000","last_peered":"2026-02-21T22:22:25.507289+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:24.696410+0000","last_became_peered":"2026-02-21T22:22:24.696410+0000","last_unstale":"2026-02-21T22:22:25.507289+0000","last_undegraded":"2026-02-21T22:22:24.456877+0000","last_fullsized":"2026-02-21T22:22:24.355844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7226,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.360463+0000","last_change":"2026-02-21T22:22:24.703206+0000","last_active":"2026-02-21T22:22:25.360463+0000","last_peered":"2026-02-21T22:22:25.360463+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:22:24.697037+0000","last_became_peered":"2026-02-21T22:22:24.697037+0000","last_unstale":"2026-02-21T22:22:25.360463+0000","last_undegraded":"2026-02-21T22:22:24.476869+0000","last_fullsized":"2026-02-21T22:22:24.355899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":27018,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:22:40.207990+0000","last_change":"2026-02-21T22:22:24.730176+0000","last_active":"2026-02-21T22:22:40.207990+0000","last_peered":"2026-02-21T22:22:40.207990+0000","last_clean":"2026-02-21T22:22:21.642353+0000","last_became_active":"2026-02-21T22:22:24.673965+0000","last_became_peered":"2026-02-21T22:22:24.673965+0000","last_unstale":"2026-02-21T22:22:40.207990+0000","last_undegraded":"2026-02-21T22:22:40.207990+0000","last_fullsized":"2026-02-21T22:22:24.356156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1040,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6617,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6617},{"shards":"2,3","objects":520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":577914,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705882,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":11,"kb_used_meta":81076,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":83021994},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":8,"kb_used_meta":81079,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8863,"internal_metadata":83025249},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:45.056 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:45.236 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:45.236 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:45.246 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-21T22:22:44.223192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162560,"kb_used_data":352,"kb_used_omap":20,"kb_used_meta":162155,"kb_avail":188581120,"statfs":{"total":193273528320,"available":193107066880,"internally_reserved":0,"allocated":360448,"data_stored":117480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20981,"internal_metadata":166047243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5844,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7287,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.555546+0000","last_change":"2026-02-21T22:22:25.556143+0000","last_active":"2026-02-21T22:22:25.555546+0000","last_peered":"2026-02-21T22:22:25.555546+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:25.552595+0000","last_became_peered":"2026-02-21T22:22:25.552595+0000","last_unstale":"2026-02-21T22:22:25.555546+0000","last_undegraded":"2026-02-21T22:22:25.506596+0000","last_fullsized":"2026-02-21T22:22:25.431530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356953+0000","last_change":"2026-02-21T22:22:24.702184+0000","last_active":"2026-02-21T22:22:25.356953+0000","last_peered":"2026-02-21T22:22:25.356953+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:22:24.697010+0000","last_became_peered":"2026-02-21T22:22:24.697010+0000","last_unstale":"2026-02-21T22:22:25.356953+0000","last_undegraded":"2026-02-21T22:22:24.474837+0000","last_fullsized":"2026-02-21T22:22:24.356051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7306,"num_objects_misplaced":7306,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7306}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7177,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.537321+0000","last_change":"2026-02-21T22:22:25.551643+0000","last_active":"2026-02-21T22:22:25.537321+0000","last_peered":"2026-02-21T22:22:25.537321+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:25.534289+0000","last_became_peered":"2026-02-21T22:22:25.534289+0000","last_unstale":"2026-02-21T22:22:25.537321+0000","last_undegraded":"2026-02-21T22:22:25.430811+0000","last_fullsized":"2026-02-21T22:22:25.357439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7313,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356967+0000","last_change":"2026-02-21T22:22:24.733636+0000","last_active":"2026-02-21T22:22:25.356967+0000","last_peered":"2026-02-21T22:22:25.356967+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:22:24.729840+0000","last_became_peered":"2026-02-21T22:22:24.729840+0000","last_unstale":"2026-02-21T22:22:25.356967+0000","last_undegraded":"2026-02-21T22:22:24.558211+0000","last_fullsized":"2026-02-21T22:22:24.477081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7258,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.510735+0000","last_change":"2026-02-21T22:22:24.702122+0000","last_active":"2026-02-21T22:22:25.510735+0000","last_peered":"2026-02-21T22:22:25.510735+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:24.697165+0000","last_became_peered":"2026-02-21T22:22:24.697165+0000","last_unstale":"2026-02-21T22:22:25.510735+0000","last_undegraded":"2026-02-21T22:22:24.528989+0000","last_fullsized":"2026-02-21T22:22:24.446698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7262,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.507289+0000","last_change":"2026-02-21T22:22:24.701417+0000","last_active":"2026-02-21T22:22:25.507289+0000","last_peered":"2026-02-21T22:22:25.507289+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:24.696410+0000","last_became_peered":"2026-02-21T22:22:24.696410+0000","last_unstale":"2026-02-21T22:22:25.507289+0000","last_undegraded":"2026-02-21T22:22:24.456877+0000","last_fullsized":"2026-02-21T22:22:24.355844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7226,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.360463+0000","last_change":"2026-02-21T22:22:24.703206+0000","last_active":"2026-02-21T22:22:25.360463+0000","last_peered":"2026-02-21T22:22:25.360463+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:22:24.697037+0000","last_became_peered":"2026-02-21T22:22:24.697037+0000","last_unstale":"2026-02-21T22:22:25.360463+0000","last_undegraded":"2026-02-21T22:22:24.476869+0000","last_fullsized":"2026-02-21T22:22:24.355899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":27018,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:22:40.207990+0000","last_change":"2026-02-21T22:22:24.730176+0000","last_active":"2026-02-21T22:22:40.207990+0000","last_peered":"2026-02-21T22:22:40.207990+0000","last_clean":"2026-02-21T22:22:21.642353+0000","last_became_active":"2026-02-21T22:22:24.673965+0000","last_became_peered":"2026-02-21T22:22:24.673965+0000","last_unstale":"2026-02-21T22:22:40.207990+0000","last_undegraded":"2026-02-21T22:22:40.207990+0000","last_fullsized":"2026-02-21T22:22:24.356156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1040,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6617,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6617},{"shards":"2,3","objects":520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":577914,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705882,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":11,"kb_used_meta":81076,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":83021994},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":8,"kb_used_meta":81079,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8863,"internal_metadata":83025249},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:45.247 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:22:45.468 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:45.469 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50578/115430 objects degraded (43.817%), 7 pgs degraded","count":7},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1070,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":23,"num_osds":4,"num_up_osds":4,"osd_up_since":1771711499,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":166461440,"bytes_avail":193107066880,"bytes_total":193273528320,"degraded_objects":50578,"degraded_total":115430,"degraded_ratio":0.43817031967426145,"misplaced_objects":51618,"misplaced_total":115430,"misplaced_ratio":0.44718010915706491,"recovering_objects_per_sec":365,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2922,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:22:45.479 INFO:tasks.osd_recovery.ceph_manager:{'fsid': 'ab2fd705-c792-40f6-9c14-b3ae92aaa530', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50578/115430 objects degraded (43.817%), 7 pgs degraded', 'count': 7}, 'muted': False}}, 'mutes': []}, 'election_epoch': 4, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 1070, 'monmap': {'epoch': 1, 'min_mon_release_name': 'tentacle', 'num_mons': 3}, 'osdmap': {'epoch': 23, 'num_osds': 4, 'num_up_osds': 4, 'osd_up_since': 1771711499, 'num_in_osds': 2, 'osd_in_since': 1771712543, 'num_remapped_pgs': 7}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+undersized+degraded+remapped', 'count': 7}, {'state_name': 'active+recovering+undersized+remapped', 'count': 1}], 'num_pgs': 8, 'num_pools': 1, 'num_objects': 57715, 'data_bytes': 346650, 'bytes_used': 166461440, 'bytes_avail': 193107066880, 'bytes_total': 193273528320, 'degraded_objects': 50578, 'degraded_total': 115430, 'degraded_ratio': 0.43817031967426145, 'misplaced_objects': 51618, 'misplaced_total': 115430, 'misplaced_ratio': 0.4471801091570649, 'recovering_objects_per_sec': 365, 'recovering_bytes_per_sec': 0, 'recovering_keys_per_sec': 0, 'num_objects_recovered': 2922, 'num_bytes_recovered': 0, 'num_keys_recovered': 0}, 'fsmap': {'epoch': 1, 'btime': '2026-02-21T22:04:55:178453+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs'], 'services': {}}, 'servicemap': {'epoch': 5, 'modified': '2026-02-21T22:21:00.210624+0000', 'services': {}}, 'progress_events': {}} 2026-02-21T22:22:45.479 INFO:tasks.osd_recovery.ceph_manager:active! 2026-02-21T22:22:45.479 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:45.663 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:45.664 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:45.673 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-21T22:22:44.223192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162560,"kb_used_data":352,"kb_used_omap":20,"kb_used_meta":162155,"kb_avail":188581120,"statfs":{"total":193273528320,"available":193107066880,"internally_reserved":0,"allocated":360448,"data_stored":117480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20981,"internal_metadata":166047243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5844,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7287,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.555546+0000","last_change":"2026-02-21T22:22:25.556143+0000","last_active":"2026-02-21T22:22:25.555546+0000","last_peered":"2026-02-21T22:22:25.555546+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:25.552595+0000","last_became_peered":"2026-02-21T22:22:25.552595+0000","last_unstale":"2026-02-21T22:22:25.555546+0000","last_undegraded":"2026-02-21T22:22:25.506596+0000","last_fullsized":"2026-02-21T22:22:25.431530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356953+0000","last_change":"2026-02-21T22:22:24.702184+0000","last_active":"2026-02-21T22:22:25.356953+0000","last_peered":"2026-02-21T22:22:25.356953+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:22:24.697010+0000","last_became_peered":"2026-02-21T22:22:24.697010+0000","last_unstale":"2026-02-21T22:22:25.356953+0000","last_undegraded":"2026-02-21T22:22:24.474837+0000","last_fullsized":"2026-02-21T22:22:24.356051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7306,"num_objects_misplaced":7306,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7306}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7177,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.537321+0000","last_change":"2026-02-21T22:22:25.551643+0000","last_active":"2026-02-21T22:22:25.537321+0000","last_peered":"2026-02-21T22:22:25.537321+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:25.534289+0000","last_became_peered":"2026-02-21T22:22:25.534289+0000","last_unstale":"2026-02-21T22:22:25.537321+0000","last_undegraded":"2026-02-21T22:22:25.430811+0000","last_fullsized":"2026-02-21T22:22:25.357439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7313,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356967+0000","last_change":"2026-02-21T22:22:24.733636+0000","last_active":"2026-02-21T22:22:25.356967+0000","last_peered":"2026-02-21T22:22:25.356967+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:22:24.729840+0000","last_became_peered":"2026-02-21T22:22:24.729840+0000","last_unstale":"2026-02-21T22:22:25.356967+0000","last_undegraded":"2026-02-21T22:22:24.558211+0000","last_fullsized":"2026-02-21T22:22:24.477081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7258,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.510735+0000","last_change":"2026-02-21T22:22:24.702122+0000","last_active":"2026-02-21T22:22:25.510735+0000","last_peered":"2026-02-21T22:22:25.510735+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:24.697165+0000","last_became_peered":"2026-02-21T22:22:24.697165+0000","last_unstale":"2026-02-21T22:22:25.510735+0000","last_undegraded":"2026-02-21T22:22:24.528989+0000","last_fullsized":"2026-02-21T22:22:24.446698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7262,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.507289+0000","last_change":"2026-02-21T22:22:24.701417+0000","last_active":"2026-02-21T22:22:25.507289+0000","last_peered":"2026-02-21T22:22:25.507289+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:24.696410+0000","last_became_peered":"2026-02-21T22:22:24.696410+0000","last_unstale":"2026-02-21T22:22:25.507289+0000","last_undegraded":"2026-02-21T22:22:24.456877+0000","last_fullsized":"2026-02-21T22:22:24.355844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7226,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.360463+0000","last_change":"2026-02-21T22:22:24.703206+0000","last_active":"2026-02-21T22:22:25.360463+0000","last_peered":"2026-02-21T22:22:25.360463+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:22:24.697037+0000","last_became_peered":"2026-02-21T22:22:24.697037+0000","last_unstale":"2026-02-21T22:22:25.360463+0000","last_undegraded":"2026-02-21T22:22:24.476869+0000","last_fullsized":"2026-02-21T22:22:24.355899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":27018,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:22:40.207990+0000","last_change":"2026-02-21T22:22:24.730176+0000","last_active":"2026-02-21T22:22:40.207990+0000","last_peered":"2026-02-21T22:22:40.207990+0000","last_clean":"2026-02-21T22:22:21.642353+0000","last_became_active":"2026-02-21T22:22:24.673965+0000","last_became_peered":"2026-02-21T22:22:24.673965+0000","last_unstale":"2026-02-21T22:22:40.207990+0000","last_undegraded":"2026-02-21T22:22:40.207990+0000","last_fullsized":"2026-02-21T22:22:24.356156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1040,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6617,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6617},{"shards":"2,3","objects":520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":577914,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705882,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":11,"kb_used_meta":81076,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":83021994},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":8,"kb_used_meta":81079,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8863,"internal_metadata":83025249},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:45.675 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:45.855 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:45.855 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:45.865 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-21T22:22:44.223192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162560,"kb_used_data":352,"kb_used_omap":20,"kb_used_meta":162155,"kb_avail":188581120,"statfs":{"total":193273528320,"available":193107066880,"internally_reserved":0,"allocated":360448,"data_stored":117480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20981,"internal_metadata":166047243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5844,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7287,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.555546+0000","last_change":"2026-02-21T22:22:25.556143+0000","last_active":"2026-02-21T22:22:25.555546+0000","last_peered":"2026-02-21T22:22:25.555546+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:25.552595+0000","last_became_peered":"2026-02-21T22:22:25.552595+0000","last_unstale":"2026-02-21T22:22:25.555546+0000","last_undegraded":"2026-02-21T22:22:25.506596+0000","last_fullsized":"2026-02-21T22:22:25.431530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356953+0000","last_change":"2026-02-21T22:22:24.702184+0000","last_active":"2026-02-21T22:22:25.356953+0000","last_peered":"2026-02-21T22:22:25.356953+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:22:24.697010+0000","last_became_peered":"2026-02-21T22:22:24.697010+0000","last_unstale":"2026-02-21T22:22:25.356953+0000","last_undegraded":"2026-02-21T22:22:24.474837+0000","last_fullsized":"2026-02-21T22:22:24.356051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7306,"num_objects_misplaced":7306,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7306}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7177,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.537321+0000","last_change":"2026-02-21T22:22:25.551643+0000","last_active":"2026-02-21T22:22:25.537321+0000","last_peered":"2026-02-21T22:22:25.537321+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:25.534289+0000","last_became_peered":"2026-02-21T22:22:25.534289+0000","last_unstale":"2026-02-21T22:22:25.537321+0000","last_undegraded":"2026-02-21T22:22:25.430811+0000","last_fullsized":"2026-02-21T22:22:25.357439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7313,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356967+0000","last_change":"2026-02-21T22:22:24.733636+0000","last_active":"2026-02-21T22:22:25.356967+0000","last_peered":"2026-02-21T22:22:25.356967+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:22:24.729840+0000","last_became_peered":"2026-02-21T22:22:24.729840+0000","last_unstale":"2026-02-21T22:22:25.356967+0000","last_undegraded":"2026-02-21T22:22:24.558211+0000","last_fullsized":"2026-02-21T22:22:24.477081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7258,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.510735+0000","last_change":"2026-02-21T22:22:24.702122+0000","last_active":"2026-02-21T22:22:25.510735+0000","last_peered":"2026-02-21T22:22:25.510735+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:24.697165+0000","last_became_peered":"2026-02-21T22:22:24.697165+0000","last_unstale":"2026-02-21T22:22:25.510735+0000","last_undegraded":"2026-02-21T22:22:24.528989+0000","last_fullsized":"2026-02-21T22:22:24.446698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7262,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.507289+0000","last_change":"2026-02-21T22:22:24.701417+0000","last_active":"2026-02-21T22:22:25.507289+0000","last_peered":"2026-02-21T22:22:25.507289+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:24.696410+0000","last_became_peered":"2026-02-21T22:22:24.696410+0000","last_unstale":"2026-02-21T22:22:25.507289+0000","last_undegraded":"2026-02-21T22:22:24.456877+0000","last_fullsized":"2026-02-21T22:22:24.355844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7226,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.360463+0000","last_change":"2026-02-21T22:22:24.703206+0000","last_active":"2026-02-21T22:22:25.360463+0000","last_peered":"2026-02-21T22:22:25.360463+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:22:24.697037+0000","last_became_peered":"2026-02-21T22:22:24.697037+0000","last_unstale":"2026-02-21T22:22:25.360463+0000","last_undegraded":"2026-02-21T22:22:24.476869+0000","last_fullsized":"2026-02-21T22:22:24.355899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":27018,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:22:40.207990+0000","last_change":"2026-02-21T22:22:24.730176+0000","last_active":"2026-02-21T22:22:40.207990+0000","last_peered":"2026-02-21T22:22:40.207990+0000","last_clean":"2026-02-21T22:22:21.642353+0000","last_became_active":"2026-02-21T22:22:24.673965+0000","last_became_peered":"2026-02-21T22:22:24.673965+0000","last_unstale":"2026-02-21T22:22:40.207990+0000","last_undegraded":"2026-02-21T22:22:40.207990+0000","last_fullsized":"2026-02-21T22:22:24.356156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1040,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6617,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6617},{"shards":"2,3","objects":520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":577914,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705882,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":11,"kb_used_meta":81076,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":83021994},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":8,"kb_used_meta":81079,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8863,"internal_metadata":83025249},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:45.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-21T22:22:45.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'7260", 'reported_seq': 7287, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.555546+0000', 'last_change': '2026-02-21T22:22:25.556143+0000', 'last_active': '2026-02-21T22:22:25.555546+0000', 'last_peered': '2026-02-21T22:22:25.555546+0000', 'last_clean': '2026-02-21T22:22:19.192165+0000', 'last_became_active': '2026-02-21T22:22:25.552595+0000', 'last_became_peered': '2026-02-21T22:22:25.552595+0000', 'last_unstale': '2026-02-21T22:22:25.555546+0000', 'last_undegraded': '2026-02-21T22:22:25.506596+0000', 'last_fullsized': '2026-02-21T22:22:25.431530+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7260, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7260, 'num_objects_misplaced': 7260, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7260}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:22:45.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:22:45.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7359, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.356953+0000', 'last_change': '2026-02-21T22:22:24.702184+0000', 'last_active': '2026-02-21T22:22:25.356953+0000', 'last_peered': '2026-02-21T22:22:25.356953+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:22:24.697010+0000', 'last_became_peered': '2026-02-21T22:22:24.697010+0000', 'last_unstale': '2026-02-21T22:22:25.356953+0000', 'last_undegraded': '2026-02-21T22:22:24.474837+0000', 'last_fullsized': '2026-02-21T22:22:24.356051+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7306, 'num_objects_misplaced': 7306, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7306}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7177, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.537321+0000', 'last_change': '2026-02-21T22:22:25.551643+0000', 'last_active': '2026-02-21T22:22:25.537321+0000', 'last_peered': '2026-02-21T22:22:25.537321+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:25.534289+0000', 'last_became_peered': '2026-02-21T22:22:25.534289+0000', 'last_unstale': '2026-02-21T22:22:25.537321+0000', 'last_undegraded': '2026-02-21T22:22:25.430811+0000', 'last_fullsized': '2026-02-21T22:22:25.357439+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 7313, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.356967+0000', 'last_change': '2026-02-21T22:22:24.733636+0000', 'last_active': '2026-02-21T22:22:25.356967+0000', 'last_peered': '2026-02-21T22:22:25.356967+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:22:24.729840+0000', 'last_became_peered': '2026-02-21T22:22:24.729840+0000', 'last_unstale': '2026-02-21T22:22:25.356967+0000', 'last_undegraded': '2026-02-21T22:22:24.558211+0000', 'last_fullsized': '2026-02-21T22:22:24.477081+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7260, 'num_objects_misplaced': 7260, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7260}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7258, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.510735+0000', 'last_change': '2026-02-21T22:22:24.702122+0000', 'last_active': '2026-02-21T22:22:25.510735+0000', 'last_peered': '2026-02-21T22:22:25.510735+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:24.697165+0000', 'last_became_peered': '2026-02-21T22:22:24.697165+0000', 'last_unstale': '2026-02-21T22:22:25.510735+0000', 'last_undegraded': '2026-02-21T22:22:24.528989+0000', 'last_fullsized': '2026-02-21T22:22:24.446698+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7262, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.507289+0000', 'last_change': '2026-02-21T22:22:24.701417+0000', 'last_active': '2026-02-21T22:22:25.507289+0000', 'last_peered': '2026-02-21T22:22:25.507289+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:24.696410+0000', 'last_became_peered': '2026-02-21T22:22:24.696410+0000', 'last_unstale': '2026-02-21T22:22:25.507289+0000', 'last_undegraded': '2026-02-21T22:22:24.456877+0000', 'last_fullsized': '2026-02-21T22:22:24.355844+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:22:45.867 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7226, 'reported_epoch': 21, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:25.360463+0000', 'last_change': '2026-02-21T22:22:24.703206+0000', 'last_active': '2026-02-21T22:22:25.360463+0000', 'last_peered': '2026-02-21T22:22:25.360463+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:22:24.697037+0000', 'last_became_peered': '2026-02-21T22:22:24.697037+0000', 'last_unstale': '2026-02-21T22:22:25.360463+0000', 'last_undegraded': '2026-02-21T22:22:24.476869+0000', 'last_fullsized': '2026-02-21T22:22:24.355899+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:22:45.868 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-21T22:22:45.868 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'7137", 'reported_seq': 27018, 'reported_epoch': 21, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:22:40.207990+0000', 'last_change': '2026-02-21T22:22:24.730176+0000', 'last_active': '2026-02-21T22:22:40.207990+0000', 'last_peered': '2026-02-21T22:22:40.207990+0000', 'last_clean': '2026-02-21T22:22:21.642353+0000', 'last_became_active': '2026-02-21T22:22:24.673965+0000', 'last_became_peered': '2026-02-21T22:22:24.673965+0000', 'last_unstale': '2026-02-21T22:22:40.207990+0000', 'last_undegraded': '2026-02-21T22:22:40.207990+0000', 'last_fullsized': '2026-02-21T22:22:24.356156+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7137, 'log_dups_size': 0, 'ondisk_log_size': 7137, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 67957, 'num_objects': 7137, 'num_object_clones': 0, 'num_object_copies': 14274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1040, 'num_objects_unfound': 0, 'num_objects_dirty': 7137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7137, 'num_write_kb': 7193, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6617, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 6617}, {'shards': '2,3', 'objects': 520}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:22:45.868 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:46.047 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:46.048 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:46.058 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-21T22:22:44.223192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162560,"kb_used_data":352,"kb_used_omap":20,"kb_used_meta":162155,"kb_avail":188581120,"statfs":{"total":193273528320,"available":193107066880,"internally_reserved":0,"allocated":360448,"data_stored":117480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20981,"internal_metadata":166047243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5844,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7287,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.555546+0000","last_change":"2026-02-21T22:22:25.556143+0000","last_active":"2026-02-21T22:22:25.555546+0000","last_peered":"2026-02-21T22:22:25.555546+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:25.552595+0000","last_became_peered":"2026-02-21T22:22:25.552595+0000","last_unstale":"2026-02-21T22:22:25.555546+0000","last_undegraded":"2026-02-21T22:22:25.506596+0000","last_fullsized":"2026-02-21T22:22:25.431530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356953+0000","last_change":"2026-02-21T22:22:24.702184+0000","last_active":"2026-02-21T22:22:25.356953+0000","last_peered":"2026-02-21T22:22:25.356953+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:22:24.697010+0000","last_became_peered":"2026-02-21T22:22:24.697010+0000","last_unstale":"2026-02-21T22:22:25.356953+0000","last_undegraded":"2026-02-21T22:22:24.474837+0000","last_fullsized":"2026-02-21T22:22:24.356051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7306,"num_objects_misplaced":7306,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7306}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7177,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.537321+0000","last_change":"2026-02-21T22:22:25.551643+0000","last_active":"2026-02-21T22:22:25.537321+0000","last_peered":"2026-02-21T22:22:25.537321+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:25.534289+0000","last_became_peered":"2026-02-21T22:22:25.534289+0000","last_unstale":"2026-02-21T22:22:25.537321+0000","last_undegraded":"2026-02-21T22:22:25.430811+0000","last_fullsized":"2026-02-21T22:22:25.357439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7313,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.356967+0000","last_change":"2026-02-21T22:22:24.733636+0000","last_active":"2026-02-21T22:22:25.356967+0000","last_peered":"2026-02-21T22:22:25.356967+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:22:24.729840+0000","last_became_peered":"2026-02-21T22:22:24.729840+0000","last_unstale":"2026-02-21T22:22:25.356967+0000","last_undegraded":"2026-02-21T22:22:24.558211+0000","last_fullsized":"2026-02-21T22:22:24.477081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7258,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.510735+0000","last_change":"2026-02-21T22:22:24.702122+0000","last_active":"2026-02-21T22:22:25.510735+0000","last_peered":"2026-02-21T22:22:25.510735+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:24.697165+0000","last_became_peered":"2026-02-21T22:22:24.697165+0000","last_unstale":"2026-02-21T22:22:25.510735+0000","last_undegraded":"2026-02-21T22:22:24.528989+0000","last_fullsized":"2026-02-21T22:22:24.446698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7262,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.507289+0000","last_change":"2026-02-21T22:22:24.701417+0000","last_active":"2026-02-21T22:22:25.507289+0000","last_peered":"2026-02-21T22:22:25.507289+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:24.696410+0000","last_became_peered":"2026-02-21T22:22:24.696410+0000","last_unstale":"2026-02-21T22:22:25.507289+0000","last_undegraded":"2026-02-21T22:22:24.456877+0000","last_fullsized":"2026-02-21T22:22:24.355844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7226,"reported_epoch":21,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:25.360463+0000","last_change":"2026-02-21T22:22:24.703206+0000","last_active":"2026-02-21T22:22:25.360463+0000","last_peered":"2026-02-21T22:22:25.360463+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:22:24.697037+0000","last_became_peered":"2026-02-21T22:22:24.697037+0000","last_unstale":"2026-02-21T22:22:25.360463+0000","last_undegraded":"2026-02-21T22:22:24.476869+0000","last_fullsized":"2026-02-21T22:22:24.355899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":27018,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:22:40.207990+0000","last_change":"2026-02-21T22:22:24.730176+0000","last_active":"2026-02-21T22:22:40.207990+0000","last_peered":"2026-02-21T22:22:40.207990+0000","last_clean":"2026-02-21T22:22:21.642353+0000","last_became_active":"2026-02-21T22:22:24.673965+0000","last_became_peered":"2026-02-21T22:22:24.673965+0000","last_unstale":"2026-02-21T22:22:40.207990+0000","last_undegraded":"2026-02-21T22:22:40.207990+0000","last_fullsized":"2026-02-21T22:22:24.356156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1040,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6617,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6617},{"shards":"2,3","objects":520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50578,"num_objects_misplaced":51618,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":577914,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705882,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":11,"kb_used_meta":81076,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":83021994},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81280,"kb_used_data":176,"kb_used_omap":8,"kb_used_meta":81079,"kb_avail":94290560,"statfs":{"total":96636764160,"available":96553533440,"internally_reserved":0,"allocated":180224,"data_stored":58740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8863,"internal_metadata":83025249},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:46.059 INFO:tasks.osd_recovery:stopping 2,3 2026-02-21T22:22:46.060 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T22:22:46.060 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:22:46.119 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T22:22:46.119 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-21T22:22:46.119 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:22:46.168 INFO:tasks.ceph.osd.3:Stopped 2026-02-21T22:22:46.169 INFO:tasks.osd_recovery:... 2026-02-21T22:22:46.169 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 3 2026-02-21T22:22:46.353 INFO:tasks.ceph.mon.a.trial060.stderr:2026-02-21T22:22:46.352+0000 7fb3dbff1640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2026-02-21T22:22:46.382 INFO:teuthology.orchestra.run.trial060.stderr:marked down osd.2. marked down osd.3. 2026-02-21T22:22:46.393 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T22:22:46.393 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T22:22:46.481 INFO:teuthology.orchestra.run.trial060.stdout:38654705885 2026-02-21T22:22:46.481 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:22:46.483 INFO:teuthology.orchestra.run.trial060.stdout:38654705884 2026-02-21T22:22:46.483 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:22:46.698 INFO:teuthology.orchestra.run.trial060.stdout:38654705884 2026-02-21T22:22:46.698 INFO:teuthology.orchestra.run.trial060.stdout:38654705883 2026-02-21T22:22:46.707 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705885 got 38654705884 for osd.1 2026-02-21T22:22:46.709 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705884 got 38654705883 for osd.0 2026-02-21T22:22:47.708 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:22:47.709 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:22:47.887 INFO:teuthology.orchestra.run.trial060.stdout:38654705884 2026-02-21T22:22:47.889 INFO:teuthology.orchestra.run.trial060.stdout:38654705883 2026-02-21T22:22:47.896 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705885 got 38654705884 for osd.1 2026-02-21T22:22:47.898 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705884 got 38654705883 for osd.0 2026-02-21T22:22:48.897 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:22:48.898 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:22:49.075 INFO:teuthology.orchestra.run.trial060.stdout:38654705884 2026-02-21T22:22:49.080 INFO:teuthology.orchestra.run.trial060.stdout:38654705885 2026-02-21T22:22:49.084 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705884 got 38654705884 for osd.0 2026-02-21T22:22:49.084 DEBUG:teuthology.parallel:result is None 2026-02-21T22:22:49.091 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705885 got 38654705885 for osd.1 2026-02-21T22:22:49.091 DEBUG:teuthology.parallel:result is None 2026-02-21T22:22:49.091 INFO:tasks.osd_recovery.ceph_manager:waiting for peering to complete or become blocked 2026-02-21T22:22:49.092 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:49.305 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:49.305 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:49.313 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2026-02-21T22:22:48.223727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7139,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162656,"kb_used_data":448,"kb_used_omap":26,"kb_used_meta":162149,"kb_avail":188581024,"statfs":{"total":193273528320,"available":193106968576,"internally_reserved":0,"allocated":458752,"data_stored":252630,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":26840,"internal_metadata":166041384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":-50578,"num_objects_misplaced":-51618,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000751"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7284,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382914+0000","last_change":"2026-02-21T22:22:46.382914+0000","last_active":"2026-02-21T22:22:46.382903+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:22:46.382914+0000","last_undegraded":"2026-02-21T22:22:46.382914+0000","last_fullsized":"2026-02-21T22:22:46.382914+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7353,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382912+0000","last_change":"2026-02-21T22:22:46.382912+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:46.382912+0000","last_undegraded":"2026-02-21T22:22:46.382912+0000","last_fullsized":"2026-02-21T22:22:46.382912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7174,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382952+0000","last_change":"2026-02-21T22:22:46.382952+0000","last_active":"2026-02-21T22:22:46.382940+0000","last_peered":"2026-02-21T22:22:22.353928+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:22:46.382952+0000","last_undegraded":"2026-02-21T22:22:46.382952+0000","last_fullsized":"2026-02-21T22:22:46.382952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":7131,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7308,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382295+0000","last_change":"2026-02-21T22:22:46.382295+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:46.382295+0000","last_undegraded":"2026-02-21T22:22:46.382295+0000","last_fullsized":"2026-02-21T22:22:46.382295+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7253,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382933+0000","last_change":"2026-02-21T22:22:46.382933+0000","last_active":"2026-02-21T22:22:46.382924+0000","last_peered":"2026-02-21T22:22:20.929230+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:22:46.382933+0000","last_undegraded":"2026-02-21T22:22:46.382933+0000","last_fullsized":"2026-02-21T22:22:46.382933+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":7220,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7256,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.383145+0000","last_change":"2026-02-21T22:22:46.383145+0000","last_active":"2026-02-21T22:22:46.383136+0000","last_peered":"2026-02-21T22:22:21.407133+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:22:46.383145+0000","last_undegraded":"2026-02-21T22:22:46.383145+0000","last_fullsized":"2026-02-21T22:22:46.383145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":7228,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7221,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382612+0000","last_change":"2026-02-21T22:22:46.382612+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:46.382612+0000","last_undegraded":"2026-02-21T22:22:46.382612+0000","last_fullsized":"2026-02-21T22:22:46.382612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7172,"reported_epoch":24,"state":"active+clean","last_fresh":"2026-02-21T22:22:46.382698+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:22:46.382698+0000","last_peered":"2026-02-21T22:22:46.382698+0000","last_clean":"2026-02-21T22:22:46.382698+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:22:46.382698+0000","last_undegraded":"2026-02-21T22:22:46.382698+0000","last_fullsized":"2026-02-21T22:22:46.382698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7139,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81328,"kb_used_data":224,"kb_used_omap":13,"kb_used_meta":81074,"kb_avail":94290512,"statfs":{"total":96636764160,"available":96553484288,"internally_reserved":0,"allocated":229376,"data_stored":126315,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":13420,"internal_metadata":83020692},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81328,"kb_used_data":224,"kb_used_omap":13,"kb_used_meta":81074,"kb_avail":94290512,"statfs":{"total":96636764160,"available":96553484288,"internally_reserved":0,"allocated":229376,"data_stored":126315,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":13420,"internal_metadata":83020692},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:49.314 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:49.490 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:49.490 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:49.498 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2026-02-21T22:22:48.223727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7139,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162656,"kb_used_data":448,"kb_used_omap":26,"kb_used_meta":162149,"kb_avail":188581024,"statfs":{"total":193273528320,"available":193106968576,"internally_reserved":0,"allocated":458752,"data_stored":252630,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":26840,"internal_metadata":166041384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":-50578,"num_objects_misplaced":-51618,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000751"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7284,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382914+0000","last_change":"2026-02-21T22:22:46.382914+0000","last_active":"2026-02-21T22:22:46.382903+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:22:46.382914+0000","last_undegraded":"2026-02-21T22:22:46.382914+0000","last_fullsized":"2026-02-21T22:22:46.382914+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7353,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382912+0000","last_change":"2026-02-21T22:22:46.382912+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:46.382912+0000","last_undegraded":"2026-02-21T22:22:46.382912+0000","last_fullsized":"2026-02-21T22:22:46.382912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7174,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382952+0000","last_change":"2026-02-21T22:22:46.382952+0000","last_active":"2026-02-21T22:22:46.382940+0000","last_peered":"2026-02-21T22:22:22.353928+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:22:46.382952+0000","last_undegraded":"2026-02-21T22:22:46.382952+0000","last_fullsized":"2026-02-21T22:22:46.382952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":7131,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7308,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382295+0000","last_change":"2026-02-21T22:22:46.382295+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:46.382295+0000","last_undegraded":"2026-02-21T22:22:46.382295+0000","last_fullsized":"2026-02-21T22:22:46.382295+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7253,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382933+0000","last_change":"2026-02-21T22:22:46.382933+0000","last_active":"2026-02-21T22:22:46.382924+0000","last_peered":"2026-02-21T22:22:20.929230+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:22:46.382933+0000","last_undegraded":"2026-02-21T22:22:46.382933+0000","last_fullsized":"2026-02-21T22:22:46.382933+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":7220,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7256,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.383145+0000","last_change":"2026-02-21T22:22:46.383145+0000","last_active":"2026-02-21T22:22:46.383136+0000","last_peered":"2026-02-21T22:22:21.407133+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:22:46.383145+0000","last_undegraded":"2026-02-21T22:22:46.383145+0000","last_fullsized":"2026-02-21T22:22:46.383145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":7228,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7221,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382612+0000","last_change":"2026-02-21T22:22:46.382612+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:46.382612+0000","last_undegraded":"2026-02-21T22:22:46.382612+0000","last_fullsized":"2026-02-21T22:22:46.382612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7172,"reported_epoch":24,"state":"active+clean","last_fresh":"2026-02-21T22:22:46.382698+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:22:46.382698+0000","last_peered":"2026-02-21T22:22:46.382698+0000","last_clean":"2026-02-21T22:22:46.382698+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:22:46.382698+0000","last_undegraded":"2026-02-21T22:22:46.382698+0000","last_fullsized":"2026-02-21T22:22:46.382698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7139,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81328,"kb_used_data":224,"kb_used_omap":13,"kb_used_meta":81074,"kb_avail":94290512,"statfs":{"total":96636764160,"available":96553484288,"internally_reserved":0,"allocated":229376,"data_stored":126315,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":13420,"internal_metadata":83020692},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81328,"kb_used_data":224,"kb_used_omap":13,"kb_used_meta":81074,"kb_avail":94290512,"statfs":{"total":96636764160,"available":96553484288,"internally_reserved":0,"allocated":229376,"data_stored":126315,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":13420,"internal_metadata":83020692},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:49.499 INFO:tasks.osd_recovery.ceph_manager:active or down! 2026-02-21T22:22:49.499 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:22:49.675 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:22:49.675 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:22:49.683 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2026-02-21T22:22:48.223727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7139,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162656,"kb_used_data":448,"kb_used_omap":26,"kb_used_meta":162149,"kb_avail":188581024,"statfs":{"total":193273528320,"available":193106968576,"internally_reserved":0,"allocated":458752,"data_stored":252630,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":26840,"internal_metadata":166041384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":-50578,"num_objects_misplaced":-51618,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000751"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7284,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382914+0000","last_change":"2026-02-21T22:22:46.382914+0000","last_active":"2026-02-21T22:22:46.382903+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:22:46.382914+0000","last_undegraded":"2026-02-21T22:22:46.382914+0000","last_fullsized":"2026-02-21T22:22:46.382914+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7353,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382912+0000","last_change":"2026-02-21T22:22:46.382912+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:46.382912+0000","last_undegraded":"2026-02-21T22:22:46.382912+0000","last_fullsized":"2026-02-21T22:22:46.382912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7174,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382952+0000","last_change":"2026-02-21T22:22:46.382952+0000","last_active":"2026-02-21T22:22:46.382940+0000","last_peered":"2026-02-21T22:22:22.353928+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:05:05.203950+0000","last_became_peered":"2026-02-21T22:05:05.203950+0000","last_unstale":"2026-02-21T22:22:46.382952+0000","last_undegraded":"2026-02-21T22:22:46.382952+0000","last_fullsized":"2026-02-21T22:22:46.382952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":7131,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7308,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382295+0000","last_change":"2026-02-21T22:22:46.382295+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:46.382295+0000","last_undegraded":"2026-02-21T22:22:46.382295+0000","last_fullsized":"2026-02-21T22:22:46.382295+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7253,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382933+0000","last_change":"2026-02-21T22:22:46.382933+0000","last_active":"2026-02-21T22:22:46.382924+0000","last_peered":"2026-02-21T22:22:20.929230+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:05:13.221015+0000","last_became_peered":"2026-02-21T22:05:13.221015+0000","last_unstale":"2026-02-21T22:22:46.382933+0000","last_undegraded":"2026-02-21T22:22:46.382933+0000","last_fullsized":"2026-02-21T22:22:46.382933+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":7220,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7256,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.383145+0000","last_change":"2026-02-21T22:22:46.383145+0000","last_active":"2026-02-21T22:22:46.383136+0000","last_peered":"2026-02-21T22:22:21.407133+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:05:13.219519+0000","last_became_peered":"2026-02-21T22:05:13.219519+0000","last_unstale":"2026-02-21T22:22:46.383145+0000","last_undegraded":"2026-02-21T22:22:46.383145+0000","last_fullsized":"2026-02-21T22:22:46.383145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":7228,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7221,"reported_epoch":24,"state":"down","last_fresh":"2026-02-21T22:22:46.382612+0000","last_change":"2026-02-21T22:22:46.382612+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:46.382612+0000","last_undegraded":"2026-02-21T22:22:46.382612+0000","last_fullsized":"2026-02-21T22:22:46.382612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7172,"reported_epoch":24,"state":"active+clean","last_fresh":"2026-02-21T22:22:46.382698+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:22:46.382698+0000","last_peered":"2026-02-21T22:22:46.382698+0000","last_clean":"2026-02-21T22:22:46.382698+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:22:46.382698+0000","last_undegraded":"2026-02-21T22:22:46.382698+0000","last_fullsized":"2026-02-21T22:22:46.382698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":48926,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7139,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81328,"kb_used_data":224,"kb_used_omap":13,"kb_used_meta":81074,"kb_avail":94290512,"statfs":{"total":96636764160,"available":96553484288,"internally_reserved":0,"allocated":229376,"data_stored":126315,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":13420,"internal_metadata":83020692},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81328,"kb_used_data":224,"kb_used_omap":13,"kb_used_meta":81074,"kb_avail":94290512,"statfs":{"total":96636764160,"available":96553484288,"internally_reserved":0,"allocated":229376,"data_stored":126315,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":13420,"internal_metadata":83020692},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:22:49.684 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T22:22:49.684 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-21T22:22:49.685 INFO:tasks.ceph.osd.2:Started 2026-02-21T22:22:49.685 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-21T22:22:49.752 INFO:teuthology.orchestra.run.trial060.stderr:no valid command found; 10 closest matches: 2026-02-21T22:22:49.752 INFO:teuthology.orchestra.run.trial060.stderr:0 2026-02-21T22:22:49.753 INFO:teuthology.orchestra.run.trial060.stderr:1 2026-02-21T22:22:49.753 INFO:teuthology.orchestra.run.trial060.stderr:2 2026-02-21T22:22:49.753 INFO:teuthology.orchestra.run.trial060.stderr:abort 2026-02-21T22:22:49.753 INFO:teuthology.orchestra.run.trial060.stderr:assert 2026-02-21T22:22:49.753 INFO:teuthology.orchestra.run.trial060.stderr:bluestore collections 2026-02-21T22:22:49.753 INFO:teuthology.orchestra.run.trial060.stderr:bluestore compression stats [] 2026-02-21T22:22:49.754 INFO:teuthology.orchestra.run.trial060.stderr:bluestore list [] [] 2026-02-21T22:22:49.754 INFO:teuthology.orchestra.run.trial060.stderr:bluestore onode metadata 2026-02-21T22:22:49.754 INFO:teuthology.orchestra.run.trial060.stderr:bluestore show sharding 2026-02-21T22:22:49.754 INFO:teuthology.orchestra.run.trial060.stderr:admin_socket: invalid command 2026-02-21T22:22:49.755 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-21T22:22:49.756 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-21T22:22:49.872 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:22:49.871+0000 7f089180b780 -1 Falling back to public interface 2026-02-21T22:22:54.521 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:22:54.520+0000 7f089180b780 -1 osd.2 23 log_to_monitors true 2026-02-21T22:22:54.757 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-21T22:22:54.824 INFO:teuthology.orchestra.run.trial060.stderr:no valid command found; 10 closest matches: 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:0 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:1 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:2 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:abort 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:assert 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:bluefs debug_inject_read_zeros 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:bluefs files list 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:bluefs stats 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:bluestore allocator dump block 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:bluestore allocator fragmentation block 2026-02-21T22:22:54.825 INFO:teuthology.orchestra.run.trial060.stderr:admin_socket: invalid command 2026-02-21T22:22:54.826 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-21T22:22:54.827 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-21T22:22:55.284 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-21T22:22:55.282+0000 7f0888b32640 -1 osd.2 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:22:59.827 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-21T22:22:59.900 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:22:59.900 INFO:teuthology.orchestra.run.trial060.stdout: "ops": [], 2026-02-21T22:22:59.900 INFO:teuthology.orchestra.run.trial060.stdout: "num_ops": 0 2026-02-21T22:22:59.900 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:22:59.906 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-21T22:22:59.906 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-21T22:22:59.908 INFO:tasks.ceph.osd.3:Started 2026-02-21T22:22:59.908 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2026-02-21T22:22:59.972 INFO:teuthology.orchestra.run.trial060.stderr:no valid command found; 10 closest matches: 2026-02-21T22:22:59.973 INFO:teuthology.orchestra.run.trial060.stderr:0 2026-02-21T22:22:59.973 INFO:teuthology.orchestra.run.trial060.stderr:1 2026-02-21T22:22:59.973 INFO:teuthology.orchestra.run.trial060.stderr:2 2026-02-21T22:22:59.973 INFO:teuthology.orchestra.run.trial060.stderr:abort 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:assert 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:bluestore collections 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:bluestore compression stats [] 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:bluestore list [] [] 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:bluestore onode metadata 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:bluestore show sharding 2026-02-21T22:22:59.974 INFO:teuthology.orchestra.run.trial060.stderr:admin_socket: invalid command 2026-02-21T22:22:59.975 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-21T22:22:59.976 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2026-02-21T22:23:00.196 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:23:00.194+0000 7f9a3da24780 -1 Falling back to public interface 2026-02-21T22:23:04.977 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2026-02-21T22:23:05.044 INFO:teuthology.orchestra.run.trial060.stderr:no valid command found; 10 closest matches: 2026-02-21T22:23:05.044 INFO:teuthology.orchestra.run.trial060.stderr:0 2026-02-21T22:23:05.044 INFO:teuthology.orchestra.run.trial060.stderr:1 2026-02-21T22:23:05.044 INFO:teuthology.orchestra.run.trial060.stderr:2 2026-02-21T22:23:05.044 INFO:teuthology.orchestra.run.trial060.stderr:abort 2026-02-21T22:23:05.044 INFO:teuthology.orchestra.run.trial060.stderr:assert 2026-02-21T22:23:05.045 INFO:teuthology.orchestra.run.trial060.stderr:bluefs debug_inject_read_zeros 2026-02-21T22:23:05.045 INFO:teuthology.orchestra.run.trial060.stderr:bluefs files list 2026-02-21T22:23:05.045 INFO:teuthology.orchestra.run.trial060.stderr:bluefs stats 2026-02-21T22:23:05.045 INFO:teuthology.orchestra.run.trial060.stderr:bluestore allocator dump block 2026-02-21T22:23:05.045 INFO:teuthology.orchestra.run.trial060.stderr:bluestore allocator fragmentation block 2026-02-21T22:23:05.045 INFO:teuthology.orchestra.run.trial060.stderr:admin_socket: invalid command 2026-02-21T22:23:05.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-21T22:23:05.046 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2026-02-21T22:23:05.142 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:23:05.140+0000 7f9a3da24780 -1 osd.3 23 log_to_monitors true 2026-02-21T22:23:05.348 INFO:tasks.ceph.osd.3.trial060.stderr:2026-02-21T22:23:05.346+0000 7f9a34def640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:23:10.048 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2026-02-21T22:23:10.118 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:23:10.119 INFO:teuthology.orchestra.run.trial060.stdout: "ops": [], 2026-02-21T22:23:10.119 INFO:teuthology.orchestra.run.trial060.stdout: "num_ops": 0 2026-02-21T22:23:10.119 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:23:10.124 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:epoch 33 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:fsid ab2fd705-c792-40f6-9c14-b3ae92aaa530 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:created 2026-02-21T22:04:55.178617+0000 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:modified 2026-02-21T22:23:08.349350+0000 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:crush_version 6 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:full_ratio 0.95 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:backfillfull_ratio 0.9 2026-02-21T22:23:10.316 INFO:teuthology.orchestra.run.trial060.stdout:nearfull_ratio 0.85 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:require_min_compat_client luminous 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:min_compat_client jewel 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:require_osd_release tentacle 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:stretch_mode_enabled false 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:max_osd 4 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:osd.0 up in weight 1 up_from 9 up_thru 31 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6805/1304182596,v1:10.20.193.60:6808/1304182596] [v2:10.20.193.60:6811/1304182596,v1:10.20.193.60:6814/1304182596] exists,up 171d695f-2570-47af-aab5-ae0bd86f3717 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:osd.1 up in weight 1 up_from 9 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6826/2501220781,v1:10.20.193.60:6827/2501220781] [v2:10.20.193.60:6828/2501220781,v1:10.20.193.60:6829/2501220781] exists,up 638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:osd.2 up out weight 0 up_from 27 up_thru 28 down_at 24 last_clean_interval [9,23) [v2:10.20.193.60:6802/4232863995,v1:10.20.193.60:6803/4232863995] [v2:10.20.193.60:6804/4232863995,v1:10.20.193.60:6806/4232863995] exists,up fd94ed65-fa21-4566-aa25-5e039e1b7d32 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:osd.3 up out weight 0 up_from 31 up_thru 32 down_at 24 last_clean_interval [9,23) [v2:10.20.193.60:6813/3051683365,v1:10.20.193.60:6815/3051683365] [v2:10.20.193.60:6816/3051683365,v1:10.20.193.60:6818/3051683365] exists,up b5d2feef-3a3b-48bd-a117-8263471172c5 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.1 [3] 2026-02-21T22:23:10.317 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.2 [2] 2026-02-21T22:23:10.318 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.3 [2] 2026-02-21T22:23:10.318 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.4 [3] 2026-02-21T22:23:10.318 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.5 [2] 2026-02-21T22:23:10.318 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.6 [3] 2026-02-21T22:23:10.318 INFO:teuthology.orchestra.run.trial060.stdout:pg_temp 1.7 [2] 2026-02-21T22:23:10.328 INFO:tasks.osd_recovery.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 31 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6805/1304182596,v1:10.20.193.60:6808/1304182596] [v2:10.20.193.60:6811/1304182596,v1:10.20.193.60:6814/1304182596] exists,up 171d695f-2570-47af-aab5-ae0bd86f3717', 'osd.1 up in weight 1 up_from 9 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:10.20.193.60:6826/2501220781,v1:10.20.193.60:6827/2501220781] [v2:10.20.193.60:6828/2501220781,v1:10.20.193.60:6829/2501220781] exists,up 638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9', 'osd.2 up out weight 0 up_from 27 up_thru 28 down_at 24 last_clean_interval [9,23) [v2:10.20.193.60:6802/4232863995,v1:10.20.193.60:6803/4232863995] [v2:10.20.193.60:6804/4232863995,v1:10.20.193.60:6806/4232863995] exists,up fd94ed65-fa21-4566-aa25-5e039e1b7d32', 'osd.3 up out weight 0 up_from 31 up_thru 32 down_at 24 last_clean_interval [9,23) [v2:10.20.193.60:6813/3051683365,v1:10.20.193.60:6815/3051683365] [v2:10.20.193.60:6816/3051683365,v1:10.20.193.60:6818/3051683365] exists,up b5d2feef-3a3b-48bd-a117-8263471172c5'] 2026-02-21T22:23:10.328 INFO:tasks.osd_recovery:all are up! 2026-02-21T22:23:10.328 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 0 2026-02-21T22:23:10.411 INFO:teuthology.orchestra.run.trial060.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T22:23:10.418 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 debug kick_recovery_wq 0 2026-02-21T22:23:10.501 INFO:teuthology.orchestra.run.trial060.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T22:23:10.507 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 0 2026-02-21T22:23:10.592 INFO:teuthology.orchestra.run.trial060.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T22:23:10.600 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 debug kick_recovery_wq 0 2026-02-21T22:23:10.683 INFO:teuthology.orchestra.run.trial060.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T22:23:10.689 INFO:tasks.osd_recovery.ceph_manager:waiting for clean 2026-02-21T22:23:10.690 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:10.872 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:10.873 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:10.882 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2026-02-21T22:23:10.226264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162784,"kb_used_data":576,"kb_used_omap":36,"kb_used_meta":162139,"kb_avail":188580896,"statfs":{"total":193273528320,"available":193106837504,"internally_reserved":0,"allocated":589824,"data_stored":308381,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":37256,"internal_metadata":166030968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.879848"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":14834,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:05.283292+0000","last_change":"2026-02-21T22:22:58.404337+0000","last_active":"2026-02-21T22:23:05.283292+0000","last_peered":"2026-02-21T22:23:05.283292+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:58.396155+0000","last_became_peered":"2026-02-21T22:22:58.396155+0000","last_unstale":"2026-02-21T22:23:05.283292+0000","last_undegraded":"2026-02-21T22:22:58.342886+0000","last_fullsized":"2026-02-21T22:22:58.288411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4748,"num_objects_misplaced":4748,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285642+0000","last_change":"2026-02-21T22:22:56.281708+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:57.285642+0000","last_undegraded":"2026-02-21T22:22:57.285642+0000","last_fullsized":"2026-02-21T22:22:57.285642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7187,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.402918+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:22:58.402918+0000","last_peered":"2026-02-21T22:22:58.402918+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:22:58.402918+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7314,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285290+0000","last_change":"2026-02-21T22:22:56.281557+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:57.285290+0000","last_undegraded":"2026-02-21T22:22:57.285290+0000","last_fullsized":"2026-02-21T22:22:57.285290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7266,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.401717+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:22:58.401717+0000","last_peered":"2026-02-21T22:22:58.401717+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:22:58.401717+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7269,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.398699+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:22:58.398699+0000","last_peered":"2026-02-21T22:22:58.398699+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:22:58.398699+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7227,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285589+0000","last_change":"2026-02-21T22:22:56.281709+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:57.285589+0000","last_undegraded":"2026-02-21T22:22:57.285589+0000","last_fullsized":"2026-02-21T22:22:57.285589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7190,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-21T22:23:08.351467+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:08.351467+0000","last_peered":"2026-02-21T22:23:08.351467+0000","last_clean":"2026-02-21T22:23:08.351467+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:08.351467+0000","last_undegraded":"2026-02-21T22:23:08.351467+0000","last_fullsized":"2026-02-21T22:23:08.351467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116995,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81376,"kb_used_data":272,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94290464,"statfs":{"total":96636764160,"available":96553435136,"internally_reserved":0,"allocated":278528,"data_stored":146974,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:10.884 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:11.065 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:11.065 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:11.075 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2026-02-21T22:23:10.226264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162784,"kb_used_data":576,"kb_used_omap":36,"kb_used_meta":162139,"kb_avail":188580896,"statfs":{"total":193273528320,"available":193106837504,"internally_reserved":0,"allocated":589824,"data_stored":308381,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":37256,"internal_metadata":166030968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.879848"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":14834,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:05.283292+0000","last_change":"2026-02-21T22:22:58.404337+0000","last_active":"2026-02-21T22:23:05.283292+0000","last_peered":"2026-02-21T22:23:05.283292+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:58.396155+0000","last_became_peered":"2026-02-21T22:22:58.396155+0000","last_unstale":"2026-02-21T22:23:05.283292+0000","last_undegraded":"2026-02-21T22:22:58.342886+0000","last_fullsized":"2026-02-21T22:22:58.288411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4748,"num_objects_misplaced":4748,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285642+0000","last_change":"2026-02-21T22:22:56.281708+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:57.285642+0000","last_undegraded":"2026-02-21T22:22:57.285642+0000","last_fullsized":"2026-02-21T22:22:57.285642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7187,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.402918+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:22:58.402918+0000","last_peered":"2026-02-21T22:22:58.402918+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:22:58.402918+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7314,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285290+0000","last_change":"2026-02-21T22:22:56.281557+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:57.285290+0000","last_undegraded":"2026-02-21T22:22:57.285290+0000","last_fullsized":"2026-02-21T22:22:57.285290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7266,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.401717+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:22:58.401717+0000","last_peered":"2026-02-21T22:22:58.401717+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:22:58.401717+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7269,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.398699+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:22:58.398699+0000","last_peered":"2026-02-21T22:22:58.398699+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:22:58.398699+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7227,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285589+0000","last_change":"2026-02-21T22:22:56.281709+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:57.285589+0000","last_undegraded":"2026-02-21T22:22:57.285589+0000","last_fullsized":"2026-02-21T22:22:57.285589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7190,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-21T22:23:08.351467+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:08.351467+0000","last_peered":"2026-02-21T22:23:08.351467+0000","last_clean":"2026-02-21T22:23:08.351467+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:08.351467+0000","last_undegraded":"2026-02-21T22:23:08.351467+0000","last_fullsized":"2026-02-21T22:23:08.351467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116995,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81376,"kb_used_data":272,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94290464,"statfs":{"total":96636764160,"available":96553435136,"internally_reserved":0,"allocated":278528,"data_stored":146974,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:11.076 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:11.261 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:11.261 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:11.271 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2026-02-21T22:23:10.226264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162784,"kb_used_data":576,"kb_used_omap":36,"kb_used_meta":162139,"kb_avail":188580896,"statfs":{"total":193273528320,"available":193106837504,"internally_reserved":0,"allocated":589824,"data_stored":308381,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":37256,"internal_metadata":166030968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.879848"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":14834,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:05.283292+0000","last_change":"2026-02-21T22:22:58.404337+0000","last_active":"2026-02-21T22:23:05.283292+0000","last_peered":"2026-02-21T22:23:05.283292+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:58.396155+0000","last_became_peered":"2026-02-21T22:22:58.396155+0000","last_unstale":"2026-02-21T22:23:05.283292+0000","last_undegraded":"2026-02-21T22:22:58.342886+0000","last_fullsized":"2026-02-21T22:22:58.288411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4748,"num_objects_misplaced":4748,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285642+0000","last_change":"2026-02-21T22:22:56.281708+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:57.285642+0000","last_undegraded":"2026-02-21T22:22:57.285642+0000","last_fullsized":"2026-02-21T22:22:57.285642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7187,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.402918+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:22:58.402918+0000","last_peered":"2026-02-21T22:22:58.402918+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:22:58.402918+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7314,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285290+0000","last_change":"2026-02-21T22:22:56.281557+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:57.285290+0000","last_undegraded":"2026-02-21T22:22:57.285290+0000","last_fullsized":"2026-02-21T22:22:57.285290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7266,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.401717+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:22:58.401717+0000","last_peered":"2026-02-21T22:22:58.401717+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:22:58.401717+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7269,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.398699+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:22:58.398699+0000","last_peered":"2026-02-21T22:22:58.398699+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:22:58.398699+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7227,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285589+0000","last_change":"2026-02-21T22:22:56.281709+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:57.285589+0000","last_undegraded":"2026-02-21T22:22:57.285589+0000","last_fullsized":"2026-02-21T22:22:57.285589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7190,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-21T22:23:08.351467+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:08.351467+0000","last_peered":"2026-02-21T22:23:08.351467+0000","last_clean":"2026-02-21T22:23:08.351467+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:08.351467+0000","last_undegraded":"2026-02-21T22:23:08.351467+0000","last_fullsized":"2026-02-21T22:23:08.351467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116995,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81376,"kb_used_data":272,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94290464,"statfs":{"total":96636764160,"available":96553435136,"internally_reserved":0,"allocated":278528,"data_stored":146974,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'7260", 'reported_seq': 14834, 'reported_epoch': 29, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:05.283292+0000', 'last_change': '2026-02-21T22:22:58.404337+0000', 'last_active': '2026-02-21T22:23:05.283292+0000', 'last_peered': '2026-02-21T22:23:05.283292+0000', 'last_clean': '2026-02-21T22:22:19.192165+0000', 'last_became_active': '2026-02-21T22:22:58.396155+0000', 'last_became_peered': '2026-02-21T22:22:58.396155+0000', 'last_unstale': '2026-02-21T22:23:05.283292+0000', 'last_undegraded': '2026-02-21T22:22:58.342886+0000', 'last_fullsized': '2026-02-21T22:22:58.288411+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7260, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4748, 'num_objects_misplaced': 4748, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2512, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 2512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7359, 'reported_epoch': 28, 'state': 'down', 'last_fresh': '2026-02-21T22:22:57.285642+0000', 'last_change': '2026-02-21T22:22:56.281708+0000', 'last_active': '2026-02-21T22:22:46.382900+0000', 'last_peered': '2026-02-21T22:22:21.168696+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:05:13.219486+0000', 'last_became_peered': '2026-02-21T22:05:13.219486+0000', 'last_unstale': '2026-02-21T22:22:57.285642+0000', 'last_undegraded': '2026-02-21T22:22:57.285642+0000', 'last_fullsized': '2026-02-21T22:22:57.285642+0000', 'mapping_epoch': 24, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 5654, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7187, 'reported_epoch': 29, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:58.402918+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:22:58.402918+0000', 'last_peered': '2026-02-21T22:22:58.402918+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:22:58.402918+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 7314, 'reported_epoch': 28, 'state': 'down', 'last_fresh': '2026-02-21T22:22:57.285290+0000', 'last_change': '2026-02-21T22:22:56.281557+0000', 'last_active': '2026-02-21T22:22:46.382283+0000', 'last_peered': '2026-02-21T22:22:21.565172+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:05:13.219955+0000', 'last_became_peered': '2026-02-21T22:05:13.219955+0000', 'last_unstale': '2026-02-21T22:22:57.285290+0000', 'last_undegraded': '2026-02-21T22:22:57.285290+0000', 'last_fullsized': '2026-02-21T22:22:57.285290+0000', 'mapping_epoch': 24, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 7260, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7266, 'reported_epoch': 29, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:58.401717+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:22:58.401717+0000', 'last_peered': '2026-02-21T22:22:58.401717+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:22:58.401717+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7269, 'reported_epoch': 29, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:22:58.398699+0000', 'last_change': '2026-02-21T22:22:58.402976+0000', 'last_active': '2026-02-21T22:22:58.398699+0000', 'last_peered': '2026-02-21T22:22:58.398699+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:22:58.398699+0000', 'last_undegraded': '2026-02-21T22:22:58.342736+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:11.272 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7227, 'reported_epoch': 28, 'state': 'down', 'last_fresh': '2026-02-21T22:22:57.285589+0000', 'last_change': '2026-02-21T22:22:56.281709+0000', 'last_active': '2026-02-21T22:22:46.382599+0000', 'last_peered': '2026-02-21T22:22:20.769088+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:05:13.220251+0000', 'last_became_peered': '2026-02-21T22:05:13.220251+0000', 'last_unstale': '2026-02-21T22:22:57.285589+0000', 'last_undegraded': '2026-02-21T22:22:57.285589+0000', 'last_fullsized': '2026-02-21T22:22:57.285589+0000', 'mapping_epoch': 24, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 7173, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2026-02-21T22:23:11.272 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:11.512 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:11.512 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 3 pgs down","count":3},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26327/115430 objects degraded (22.808%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1096,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":33,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"down","count":3},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+clean","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":166690816,"bytes_avail":193106837504,"bytes_total":193273528320,"inactive_pgs_ratio":0.375,"degraded_objects":26327,"degraded_total":115430,"degraded_ratio":0.22807762280169799,"misplaced_objects":26327,"misplaced_total":115430,"misplaced_ratio":0.22807762280169799},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:11.525 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:23:11.525 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:11.710 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:11.711 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:11.721 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2026-02-21T22:23:10.226264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162784,"kb_used_data":576,"kb_used_omap":36,"kb_used_meta":162139,"kb_avail":188580896,"statfs":{"total":193273528320,"available":193106837504,"internally_reserved":0,"allocated":589824,"data_stored":308381,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":37256,"internal_metadata":166030968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.879848"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":14834,"reported_epoch":29,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:05.283292+0000","last_change":"2026-02-21T22:22:58.404337+0000","last_active":"2026-02-21T22:23:05.283292+0000","last_peered":"2026-02-21T22:23:05.283292+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:58.396155+0000","last_became_peered":"2026-02-21T22:22:58.396155+0000","last_unstale":"2026-02-21T22:23:05.283292+0000","last_undegraded":"2026-02-21T22:22:58.342886+0000","last_fullsized":"2026-02-21T22:22:58.288411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4748,"num_objects_misplaced":4748,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7359,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285642+0000","last_change":"2026-02-21T22:22:56.281708+0000","last_active":"2026-02-21T22:22:46.382900+0000","last_peered":"2026-02-21T22:22:21.168696+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:05:13.219486+0000","last_became_peered":"2026-02-21T22:05:13.219486+0000","last_unstale":"2026-02-21T22:22:57.285642+0000","last_undegraded":"2026-02-21T22:22:57.285642+0000","last_fullsized":"2026-02-21T22:22:57.285642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":5654,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7187,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.402918+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:22:58.402918+0000","last_peered":"2026-02-21T22:22:58.402918+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:22:58.402918+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7314,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285290+0000","last_change":"2026-02-21T22:22:56.281557+0000","last_active":"2026-02-21T22:22:46.382283+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:22:57.285290+0000","last_undegraded":"2026-02-21T22:22:57.285290+0000","last_fullsized":"2026-02-21T22:22:57.285290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":7260,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7266,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.401717+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:22:58.401717+0000","last_peered":"2026-02-21T22:22:58.401717+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:22:58.401717+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7269,"reported_epoch":29,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:22:58.398699+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:22:58.398699+0000","last_peered":"2026-02-21T22:22:58.398699+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:22:58.398699+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7227,"reported_epoch":28,"state":"down","last_fresh":"2026-02-21T22:22:57.285589+0000","last_change":"2026-02-21T22:22:56.281709+0000","last_active":"2026-02-21T22:22:46.382599+0000","last_peered":"2026-02-21T22:22:20.769088+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:05:13.220251+0000","last_became_peered":"2026-02-21T22:05:13.220251+0000","last_unstale":"2026-02-21T22:22:57.285589+0000","last_undegraded":"2026-02-21T22:22:57.285589+0000","last_fullsized":"2026-02-21T22:22:57.285589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":7173,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7190,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-21T22:23:08.351467+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:08.351467+0000","last_peered":"2026-02-21T22:23:08.351467+0000","last_clean":"2026-02-21T22:23:08.351467+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:08.351467+0000","last_undegraded":"2026-02-21T22:23:08.351467+0000","last_fullsized":"2026-02-21T22:23:08.351467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":20087,"num_objects_degraded":26327,"num_objects_misplaced":26327,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9651,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116995,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81376,"kb_used_data":272,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94290464,"statfs":{"total":96636764160,"available":96553435136,"internally_reserved":0,"allocated":278528,"data_stored":146974,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:14.723 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:14.905 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:14.905 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:14.915 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2026-02-21T22:23:14.362246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41666,"num_objects_misplaced":46710,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11877,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":9,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162816,"kb_used_data":608,"kb_used_omap":38,"kb_used_meta":162137,"kb_avail":188580864,"statfs":{"total":193273528320,"available":193106804736,"internally_reserved":0,"allocated":622592,"data_stored":322814,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":39209,"internal_metadata":166029015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-20087,"num_objects_degraded":15339,"num_objects_misplaced":20383,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.012458"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":21520,"reported_epoch":33,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:10.283676+0000","last_change":"2026-02-21T22:23:06.420776+0000","last_active":"2026-02-21T22:23:10.283676+0000","last_peered":"2026-02-21T22:23:10.283676+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:58.396155+0000","last_became_peered":"2026-02-21T22:22:58.396155+0000","last_unstale":"2026-02-21T22:23:10.283676+0000","last_undegraded":"2026-02-21T22:23:10.283676+0000","last_fullsized":"2026-02-21T22:22:58.288411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5044,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4738,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4738},{"shards":"2,3","objects":2522}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7366,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.478123+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:08.478123+0000","last_peered":"2026-02-21T22:23:08.478123+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:08.478123+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7195,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.354597+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:08.354597+0000","last_peered":"2026-02-21T22:23:08.354597+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:08.354597+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7321,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.454703+0000","last_change":"2026-02-21T22:23:08.473212+0000","last_active":"2026-02-21T22:23:08.454703+0000","last_peered":"2026-02-21T22:23:08.454703+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:08.454703+0000","last_undegraded":"2026-02-21T22:23:08.410668+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7274,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.353027+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:08.353027+0000","last_peered":"2026-02-21T22:23:08.353027+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:08.353027+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7277,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.351435+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:08.351435+0000","last_peered":"2026-02-21T22:23:08.351435+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:08.351435+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7234,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.477552+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:08.477552+0000","last_peered":"2026-02-21T22:23:08.477552+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:08.477552+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7190,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-21T22:23:08.351467+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:08.351467+0000","last_peered":"2026-02-21T22:23:08.351467+0000","last_clean":"2026-02-21T22:23:08.351467+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:08.351467+0000","last_undegraded":"2026-02-21T22:23:08.351467+0000","last_fullsized":"2026-02-21T22:23:08.351467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41666,"num_objects_misplaced":46710,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11877,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":9,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986178,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116996,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":83014833},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:14.916 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:15.092 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:15.093 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:15.103 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2026-02-21T22:23:14.362246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41666,"num_objects_misplaced":46710,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11877,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":9,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":162816,"kb_used_data":608,"kb_used_omap":38,"kb_used_meta":162137,"kb_avail":188580864,"statfs":{"total":193273528320,"available":193106804736,"internally_reserved":0,"allocated":622592,"data_stored":322814,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":39209,"internal_metadata":166029015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-20087,"num_objects_degraded":15339,"num_objects_misplaced":20383,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.012458"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":21520,"reported_epoch":33,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:10.283676+0000","last_change":"2026-02-21T22:23:06.420776+0000","last_active":"2026-02-21T22:23:10.283676+0000","last_peered":"2026-02-21T22:23:10.283676+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:22:58.396155+0000","last_became_peered":"2026-02-21T22:22:58.396155+0000","last_unstale":"2026-02-21T22:23:10.283676+0000","last_undegraded":"2026-02-21T22:23:10.283676+0000","last_fullsized":"2026-02-21T22:22:58.288411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5044,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4738,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4738},{"shards":"2,3","objects":2522}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7366,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.478123+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:08.478123+0000","last_peered":"2026-02-21T22:23:08.478123+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:08.478123+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7195,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.354597+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:08.354597+0000","last_peered":"2026-02-21T22:23:08.354597+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:08.354597+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7321,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.454703+0000","last_change":"2026-02-21T22:23:08.473212+0000","last_active":"2026-02-21T22:23:08.454703+0000","last_peered":"2026-02-21T22:23:08.454703+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:08.454703+0000","last_undegraded":"2026-02-21T22:23:08.410668+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7260}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7274,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.353027+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:08.353027+0000","last_peered":"2026-02-21T22:23:08.353027+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:08.353027+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7277,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.351435+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:08.351435+0000","last_peered":"2026-02-21T22:23:08.351435+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:08.351435+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7234,"reported_epoch":33,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:08.477552+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:08.477552+0000","last_peered":"2026-02-21T22:23:08.477552+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:08.477552+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7190,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-21T22:23:08.351467+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:08.351467+0000","last_peered":"2026-02-21T22:23:08.351467+0000","last_clean":"2026-02-21T22:23:08.351467+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:08.351467+0000","last_undegraded":"2026-02-21T22:23:08.351467+0000","last_fullsized":"2026-02-21T22:23:08.351467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41666,"num_objects_misplaced":46710,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11877,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":9,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986178,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116996,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81408,"kb_used_data":304,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94290432,"statfs":{"total":96636764160,"available":96553402368,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":83014833},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:15.104 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-21T22:23:15.104 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'7260", 'reported_seq': 21520, 'reported_epoch': 33, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:10.283676+0000', 'last_change': '2026-02-21T22:23:06.420776+0000', 'last_active': '2026-02-21T22:23:10.283676+0000', 'last_peered': '2026-02-21T22:23:10.283676+0000', 'last_clean': '2026-02-21T22:22:19.192165+0000', 'last_became_active': '2026-02-21T22:22:58.396155+0000', 'last_became_peered': '2026-02-21T22:22:58.396155+0000', 'last_unstale': '2026-02-21T22:23:10.283676+0000', 'last_undegraded': '2026-02-21T22:23:10.283676+0000', 'last_fullsized': '2026-02-21T22:22:58.288411+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7260, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 5044, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4738, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 4738}, {'shards': '2,3', 'objects': 2522}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:15.105 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:15.105 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7366, 'reported_epoch': 33, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:08.478123+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:08.478123+0000', 'last_peered': '2026-02-21T22:23:08.478123+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:08.478123+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:15.105 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:15.105 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7195, 'reported_epoch': 33, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:08.354597+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:08.354597+0000', 'last_peered': '2026-02-21T22:23:08.354597+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:08.354597+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:15.105 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:15.105 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 7321, 'reported_epoch': 33, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:08.454703+0000', 'last_change': '2026-02-21T22:23:08.473212+0000', 'last_active': '2026-02-21T22:23:08.454703+0000', 'last_peered': '2026-02-21T22:23:08.454703+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:23:08.452135+0000', 'last_became_peered': '2026-02-21T22:23:08.452135+0000', 'last_unstale': '2026-02-21T22:23:08.454703+0000', 'last_undegraded': '2026-02-21T22:23:08.410668+0000', 'last_fullsized': '2026-02-21T22:23:08.351210+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7260, 'num_objects_misplaced': 7260, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7260}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:15.106 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:15.106 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7274, 'reported_epoch': 33, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:08.353027+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:08.353027+0000', 'last_peered': '2026-02-21T22:23:08.353027+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:08.353027+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:15.106 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:15.106 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7277, 'reported_epoch': 33, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:08.351435+0000', 'last_change': '2026-02-21T22:22:58.402976+0000', 'last_active': '2026-02-21T22:23:08.351435+0000', 'last_peered': '2026-02-21T22:23:08.351435+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:08.351435+0000', 'last_undegraded': '2026-02-21T22:22:58.342736+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:15.107 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:15.107 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7234, 'reported_epoch': 33, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:08.477552+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:08.477552+0000', 'last_peered': '2026-02-21T22:23:08.477552+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:08.477552+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:15.107 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:15.333 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:15.333 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41666/115430 objects degraded (36.096%), 6 pgs degraded","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1100,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":34,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+clean","count":1},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":166723584,"bytes_avail":193106804736,"bytes_total":193273528320,"degraded_objects":41666,"degraded_total":115430,"degraded_ratio":0.36096335441393052,"misplaced_objects":46710,"misplaced_total":115430,"misplaced_ratio":0.40466083340552716,"recovering_objects_per_sec":282,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2226,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:15.344 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:15.344 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:15.527 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:15.527 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:15.537 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":580,"stamp":"2026-02-21T22:23:15.363123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":198920,"kb_used_data":616,"kb_used_omap":38,"kb_used_meta":198233,"kb_avail":188544760,"statfs":{"total":193273528320,"available":193069834240,"internally_reserved":0,"allocated":630784,"data_stored":326175,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":39209,"internal_metadata":202991319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-20087,"num_objects_degraded":8079,"num_objects_misplaced":21699,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.137091"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7289,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-21T22:23:14.363255+0000","last_change":"2026-02-21T22:23:14.363255+0000","last_active":"2026-02-21T22:23:14.363255+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:23:14.363255+0000","last_undegraded":"2026-02-21T22:23:14.363255+0000","last_fullsized":"2026-02-21T22:23:14.363255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7368,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.365905+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:14.365905+0000","last_peered":"2026-02-21T22:23:14.365905+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:14.365905+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7197,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.366244+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:14.366244+0000","last_peered":"2026-02-21T22:23:14.366244+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:14.366244+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":8674,"reported_epoch":34,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:15.349629+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:15.349629+0000","last_peered":"2026-02-21T22:23:15.349629+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:15.349629+0000","last_undegraded":"2026-02-21T22:23:15.349629+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13620,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":6810},{"shards":"3","objects":450}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7276,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.363569+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:14.363569+0000","last_peered":"2026-02-21T22:23:14.363569+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:14.363569+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7279,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362930+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:14.362930+0000","last_peered":"2026-02-21T22:23:14.362930+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:14.362930+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7236,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362925+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:14.362925+0000","last_peered":"2026-02-21T22:23:14.362925+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:14.362925+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7192,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-21T22:23:14.363494+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:14.363494+0000","last_peered":"2026-02-21T22:23:14.363494+0000","last_clean":"2026-02-21T22:23:14.363494+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:14.363494+0000","last_undegraded":"2026-02-21T22:23:14.363494+0000","last_fullsized":"2026-02-21T22:23:14.363494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116997,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99456,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":99116,"kb_avail":94272384,"statfs":{"total":96636764160,"available":96534921216,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":101495334},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99464,"kb_used_data":312,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":94272376,"statfs":{"total":96636764160,"available":96534913024,"internally_reserved":0,"allocated":319488,"data_stored":164768,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":101495985},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:18.539 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:18.722 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:18.723 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:18.732 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":582,"stamp":"2026-02-21T22:23:18.227230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":198920,"kb_used_data":616,"kb_used_omap":38,"kb_used_meta":198233,"kb_avail":188544760,"statfs":{"total":193273528320,"available":193069834240,"internally_reserved":0,"allocated":630784,"data_stored":326175,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":39209,"internal_metadata":202991319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-20087,"num_objects_degraded":8079,"num_objects_misplaced":21699,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7289,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-21T22:23:14.363255+0000","last_change":"2026-02-21T22:23:14.363255+0000","last_active":"2026-02-21T22:23:14.363255+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:23:14.363255+0000","last_undegraded":"2026-02-21T22:23:14.363255+0000","last_fullsized":"2026-02-21T22:23:14.363255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7368,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.365905+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:14.365905+0000","last_peered":"2026-02-21T22:23:14.365905+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:14.365905+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7197,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.366244+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:14.366244+0000","last_peered":"2026-02-21T22:23:14.366244+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:14.366244+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":8674,"reported_epoch":34,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:15.349629+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:15.349629+0000","last_peered":"2026-02-21T22:23:15.349629+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:15.349629+0000","last_undegraded":"2026-02-21T22:23:15.349629+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13620,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":6810},{"shards":"3","objects":450}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7276,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.363569+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:14.363569+0000","last_peered":"2026-02-21T22:23:14.363569+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:14.363569+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7279,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362930+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:14.362930+0000","last_peered":"2026-02-21T22:23:14.362930+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:14.362930+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7236,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362925+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:14.362925+0000","last_peered":"2026-02-21T22:23:14.362925+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:14.362925+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7192,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-21T22:23:14.363494+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:14.363494+0000","last_peered":"2026-02-21T22:23:14.363494+0000","last_clean":"2026-02-21T22:23:14.363494+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:14.363494+0000","last_undegraded":"2026-02-21T22:23:14.363494+0000","last_fullsized":"2026-02-21T22:23:14.363494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116997,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99456,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":99116,"kb_avail":94272384,"statfs":{"total":96636764160,"available":96534921216,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":101495334},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99464,"kb_used_data":312,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":94272376,"statfs":{"total":96636764160,"available":96534913024,"internally_reserved":0,"allocated":319488,"data_stored":164768,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":101495985},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:18.733 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:18.915 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:18.915 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:18.925 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":582,"stamp":"2026-02-21T22:23:18.227230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":198920,"kb_used_data":616,"kb_used_omap":38,"kb_used_meta":198233,"kb_avail":188544760,"statfs":{"total":193273528320,"available":193069834240,"internally_reserved":0,"allocated":630784,"data_stored":326175,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":39209,"internal_metadata":202991319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-20087,"num_objects_degraded":8079,"num_objects_misplaced":21699,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7289,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-21T22:23:14.363255+0000","last_change":"2026-02-21T22:23:14.363255+0000","last_active":"2026-02-21T22:23:14.363255+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:23:14.363255+0000","last_undegraded":"2026-02-21T22:23:14.363255+0000","last_fullsized":"2026-02-21T22:23:14.363255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7368,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.365905+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:14.365905+0000","last_peered":"2026-02-21T22:23:14.365905+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:14.365905+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7197,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.366244+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:14.366244+0000","last_peered":"2026-02-21T22:23:14.366244+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:14.366244+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":8674,"reported_epoch":34,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:15.349629+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:15.349629+0000","last_peered":"2026-02-21T22:23:15.349629+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:15.349629+0000","last_undegraded":"2026-02-21T22:23:15.349629+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13620,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":6810},{"shards":"3","objects":450}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7276,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.363569+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:14.363569+0000","last_peered":"2026-02-21T22:23:14.363569+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:14.363569+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7279,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362930+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:14.362930+0000","last_peered":"2026-02-21T22:23:14.362930+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:14.362930+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7236,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362925+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:14.362925+0000","last_peered":"2026-02-21T22:23:14.362925+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:14.362925+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7192,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-21T22:23:14.363494+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:14.363494+0000","last_peered":"2026-02-21T22:23:14.363494+0000","last_clean":"2026-02-21T22:23:14.363494+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:14.363494+0000","last_undegraded":"2026-02-21T22:23:14.363494+0000","last_fullsized":"2026-02-21T22:23:14.363494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116997,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99456,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":99116,"kb_avail":94272384,"statfs":{"total":96636764160,"available":96534921216,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":101495334},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99464,"kb_used_data":312,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":94272376,"statfs":{"total":96636764160,"available":96534913024,"internally_reserved":0,"allocated":319488,"data_stored":164768,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":101495985},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'7260", 'reported_seq': 7289, 'reported_epoch': 34, 'state': 'peering', 'last_fresh': '2026-02-21T22:23:14.363255+0000', 'last_change': '2026-02-21T22:23:14.363255+0000', 'last_active': '2026-02-21T22:23:14.363255+0000', 'last_peered': '2026-02-21T22:22:19.192165+0000', 'last_clean': '2026-02-21T22:22:19.192165+0000', 'last_became_active': '2026-02-21T22:05:13.219857+0000', 'last_became_peered': '2026-02-21T22:05:13.219857+0000', 'last_unstale': '2026-02-21T22:23:14.363255+0000', 'last_undegraded': '2026-02-21T22:23:14.363255+0000', 'last_fullsized': '2026-02-21T22:23:14.363255+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7260, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7368, 'reported_epoch': 34, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:14.365905+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:14.365905+0000', 'last_peered': '2026-02-21T22:23:14.365905+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:14.365905+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7197, 'reported_epoch': 34, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:14.366244+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:14.366244+0000', 'last_peered': '2026-02-21T22:23:14.366244+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:14.366244+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 8674, 'reported_epoch': 34, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:15.349629+0000', 'last_change': '2026-02-21T22:23:13.940568+0000', 'last_active': '2026-02-21T22:23:15.349629+0000', 'last_peered': '2026-02-21T22:23:15.349629+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:23:08.452135+0000', 'last_became_peered': '2026-02-21T22:23:08.452135+0000', 'last_unstale': '2026-02-21T22:23:15.349629+0000', 'last_undegraded': '2026-02-21T22:23:15.349629+0000', 'last_fullsized': '2026-02-21T22:23:08.351210+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 13620, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 450, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 6810}, {'shards': '3', 'objects': 450}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7276, 'reported_epoch': 34, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:14.363569+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:14.363569+0000', 'last_peered': '2026-02-21T22:23:14.363569+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:14.363569+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7279, 'reported_epoch': 34, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:14.362930+0000', 'last_change': '2026-02-21T22:22:58.402976+0000', 'last_active': '2026-02-21T22:23:14.362930+0000', 'last_peered': '2026-02-21T22:23:14.362930+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:14.362930+0000', 'last_undegraded': '2026-02-21T22:22:58.342736+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:18.926 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:18.927 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7236, 'reported_epoch': 34, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:14.362925+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:14.362925+0000', 'last_peered': '2026-02-21T22:23:14.362925+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:14.362925+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:18.927 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:19.146 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:19.146 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34406/115430 objects degraded (29.807%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1103,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":35,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+remapped","count":1},{"state_name":"active+clean","count":1},{"state_name":"peering","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":203694080,"bytes_avail":193069834240,"bytes_total":193273528320,"inactive_pgs_ratio":0.125,"degraded_objects":34406,"degraded_total":115430,"degraded_ratio":0.29806809321666811,"misplaced_objects":48026,"misplaced_total":115430,"misplaced_ratio":0.41606168240492075},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:19.157 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-21T22:23:19.157 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:19.340 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:19.341 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:19.351 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":582,"stamp":"2026-02-21T22:23:18.227230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":198920,"kb_used_data":616,"kb_used_omap":38,"kb_used_meta":198233,"kb_avail":188544760,"statfs":{"total":193273528320,"available":193069834240,"internally_reserved":0,"allocated":630784,"data_stored":326175,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":39209,"internal_metadata":202991319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-20087,"num_objects_degraded":8079,"num_objects_misplaced":21699,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7289,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-21T22:23:14.363255+0000","last_change":"2026-02-21T22:23:14.363255+0000","last_active":"2026-02-21T22:23:14.363255+0000","last_peered":"2026-02-21T22:22:19.192165+0000","last_clean":"2026-02-21T22:22:19.192165+0000","last_became_active":"2026-02-21T22:05:13.219857+0000","last_became_peered":"2026-02-21T22:05:13.219857+0000","last_unstale":"2026-02-21T22:23:14.363255+0000","last_undegraded":"2026-02-21T22:23:14.363255+0000","last_fullsized":"2026-02-21T22:23:14.363255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7368,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.365905+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:14.365905+0000","last_peered":"2026-02-21T22:23:14.365905+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:14.365905+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7197,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.366244+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:14.366244+0000","last_peered":"2026-02-21T22:23:14.366244+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:14.366244+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":8674,"reported_epoch":34,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:15.349629+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:15.349629+0000","last_peered":"2026-02-21T22:23:15.349629+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:15.349629+0000","last_undegraded":"2026-02-21T22:23:15.349629+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13620,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":6810},{"shards":"3","objects":450}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7276,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.363569+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:14.363569+0000","last_peered":"2026-02-21T22:23:14.363569+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:14.363569+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7279,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362930+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:14.362930+0000","last_peered":"2026-02-21T22:23:14.362930+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:14.362930+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7236,"reported_epoch":34,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:14.362925+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:14.362925+0000","last_peered":"2026-02-21T22:23:14.362925+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:14.362925+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7192,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-21T22:23:14.363494+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:14.363494+0000","last_peered":"2026-02-21T22:23:14.363494+0000","last_clean":"2026-02-21T22:23:14.363494+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:14.363494+0000","last_undegraded":"2026-02-21T22:23:14.363494+0000","last_fullsized":"2026-02-21T22:23:14.363494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":48026,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7589,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116997,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99456,"kb_used_data":304,"kb_used_omap":19,"kb_used_meta":99116,"kb_avail":94272384,"statfs":{"total":96636764160,"available":96534921216,"internally_reserved":0,"allocated":311296,"data_stored":161407,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19930,"internal_metadata":101495334},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99464,"kb_used_data":312,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":94272376,"statfs":{"total":96636764160,"available":96534913024,"internally_reserved":0,"allocated":319488,"data_stored":164768,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":101495985},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:22.353 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:22.532 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:22.532 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:22.542 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":584,"stamp":"2026-02-21T22:23:22.227633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":44556,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7260,"num_objects_misplaced":-2154,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4707,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000893"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":13881,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:20.350262+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:20.350262+0000","last_peered":"2026-02-21T22:23:20.350262+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:20.350262+0000","last_undegraded":"2026-02-21T22:23:20.350262+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10150,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5075},{"shards":"3","objects":2185}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":44556,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986180,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116998,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:22.543 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:22.728 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:22.728 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:22.738 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":584,"stamp":"2026-02-21T22:23:22.227633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":44556,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7260,"num_objects_misplaced":-2154,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4707,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000893"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":13881,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:20.350262+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:20.350262+0000","last_peered":"2026-02-21T22:23:20.350262+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:20.350262+0000","last_undegraded":"2026-02-21T22:23:20.350262+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10150,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5075},{"shards":"3","objects":2185}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":44556,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986180,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116998,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:22.738 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:22.738 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7370, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.367583+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:15.367583+0000', 'last_peered': '2026-02-21T22:23:15.367583+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:15.367583+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7199, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.367570+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:15.367570+0000', 'last_peered': '2026-02-21T22:23:15.367570+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:15.367570+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 13881, 'reported_epoch': 35, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:20.350262+0000', 'last_change': '2026-02-21T22:23:13.940568+0000', 'last_active': '2026-02-21T22:23:20.350262+0000', 'last_peered': '2026-02-21T22:23:20.350262+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:23:08.452135+0000', 'last_became_peered': '2026-02-21T22:23:08.452135+0000', 'last_unstale': '2026-02-21T22:23:20.350262+0000', 'last_undegraded': '2026-02-21T22:23:20.350262+0000', 'last_fullsized': '2026-02-21T22:23:08.351210+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 10150, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2185, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5075}, {'shards': '3', 'objects': 2185}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7278, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363914+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:15.363914+0000', 'last_peered': '2026-02-21T22:23:15.363914+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:15.363914+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7281, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363776+0000', 'last_change': '2026-02-21T22:22:58.402976+0000', 'last_active': '2026-02-21T22:23:15.363776+0000', 'last_peered': '2026-02-21T22:23:15.363776+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:15.363776+0000', 'last_undegraded': '2026-02-21T22:22:58.342736+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:22.739 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7238, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363627+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:15.363627+0000', 'last_peered': '2026-02-21T22:23:15.363627+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:15.363627+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:22.739 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:22.958 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:22.958 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34406/115430 objects degraded (29.807%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1107,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":35,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+clean","count":2},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240680960,"bytes_avail":193032847360,"bytes_total":193273528320,"degraded_objects":34406,"degraded_total":115430,"degraded_ratio":0.29806809321666811,"misplaced_objects":44556,"misplaced_total":115430,"misplaced_ratio":0.38600017326518243,"recovering_objects_per_sec":588,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":4707,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:22.968 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:22.968 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:23.145 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:23.146 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:23.155 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":584,"stamp":"2026-02-21T22:23:22.227633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":44556,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7260,"num_objects_misplaced":-2154,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4707,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000893"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":13881,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:20.350262+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:20.350262+0000","last_peered":"2026-02-21T22:23:20.350262+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:20.350262+0000","last_undegraded":"2026-02-21T22:23:20.350262+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10150,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5075},{"shards":"3","objects":2185}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":44556,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986180,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116998,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:26.157 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:26.334 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:26.334 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:26.344 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2026-02-21T22:23:26.228131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7746,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.865008"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":20295,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:25.352296+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:25.352296+0000","last_peered":"2026-02-21T22:23:25.352296+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:25.352296+0000","last_undegraded":"2026-02-21T22:23:25.352296+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5874,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2937},{"shards":"3","objects":4323}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116999,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:26.346 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:26.523 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:26.523 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:26.532 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2026-02-21T22:23:26.228131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7746,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.865008"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":20295,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:25.352296+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:25.352296+0000","last_peered":"2026-02-21T22:23:25.352296+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:25.352296+0000","last_undegraded":"2026-02-21T22:23:25.352296+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5874,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2937},{"shards":"3","objects":4323}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116999,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:26.534 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:26.534 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7370, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.367583+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:15.367583+0000', 'last_peered': '2026-02-21T22:23:15.367583+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:15.367583+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:26.534 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:26.534 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7199, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.367570+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:15.367570+0000', 'last_peered': '2026-02-21T22:23:15.367570+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:15.367570+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:26.534 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:26.534 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 20295, 'reported_epoch': 35, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:25.352296+0000', 'last_change': '2026-02-21T22:23:13.940568+0000', 'last_active': '2026-02-21T22:23:25.352296+0000', 'last_peered': '2026-02-21T22:23:25.352296+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:23:08.452135+0000', 'last_became_peered': '2026-02-21T22:23:08.452135+0000', 'last_unstale': '2026-02-21T22:23:25.352296+0000', 'last_undegraded': '2026-02-21T22:23:25.352296+0000', 'last_fullsized': '2026-02-21T22:23:08.351210+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 5874, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4323, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 2937}, {'shards': '3', 'objects': 4323}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:26.535 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:26.535 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7278, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363914+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:15.363914+0000', 'last_peered': '2026-02-21T22:23:15.363914+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:15.363914+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:26.535 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:26.535 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7281, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363776+0000', 'last_change': '2026-02-21T22:22:58.402976+0000', 'last_active': '2026-02-21T22:23:15.363776+0000', 'last_peered': '2026-02-21T22:23:15.363776+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:15.363776+0000', 'last_undegraded': '2026-02-21T22:22:58.342736+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:26.535 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:26.535 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7238, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363627+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:15.363627+0000', 'last_peered': '2026-02-21T22:23:15.363627+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:15.363627+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:26.536 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:26.759 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:26.759 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34406/115430 objects degraded (29.807%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1111,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":35,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+clean","count":2},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240680960,"bytes_avail":193032847360,"bytes_total":193273528320,"degraded_objects":34406,"degraded_total":115430,"degraded_ratio":0.29806809321666811,"misplaced_objects":40280,"misplaced_total":115430,"misplaced_ratio":0.34895607727627131,"recovering_objects_per_sec":1024,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":11133,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:26.769 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:26.770 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:26.951 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:26.952 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:26.962 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2026-02-21T22:23:26.228131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7746,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.865008"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":20295,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:25.352296+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:25.352296+0000","last_peered":"2026-02-21T22:23:25.352296+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:25.352296+0000","last_undegraded":"2026-02-21T22:23:25.352296+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5874,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2937},{"shards":"3","objects":4323}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116999,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:29.964 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:30.144 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:30.145 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:30.154 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":588,"stamp":"2026-02-21T22:23:29.854252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235040,"kb_used_data":640,"kb_used_omap":39,"kb_used_meta":234328,"kb_avail":188508640,"statfs":{"total":193273528320,"available":193032847360,"internally_reserved":0,"allocated":655360,"data_stored":336216,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":40511,"internal_metadata":239952321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7746,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.627021"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7298,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.368303+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:15.368303+0000","last_peered":"2026-02-21T22:23:15.368303+0000","last_clean":"2026-02-21T22:23:15.368303+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:15.368303+0000","last_undegraded":"2026-02-21T22:23:15.368303+0000","last_fullsized":"2026-02-21T22:23:15.368303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":20295,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:25.352296+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:25.352296+0000","last_peered":"2026-02-21T22:23:25.352296+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:25.352296+0000","last_undegraded":"2026-02-21T22:23:25.352296+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5874,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2937},{"shards":"3","objects":4323}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7194,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-21T22:23:15.363702+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:15.363702+0000","last_peered":"2026-02-21T22:23:15.363702+0000","last_clean":"2026-02-21T22:23:15.363702+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:15.363702+0000","last_undegraded":"2026-02-21T22:23:15.363702+0000","last_fullsized":"2026-02-21T22:23:15.363702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":699562,"data_compressed":176502,"data_compressed_allocated":188416,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116999,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":20,"kb_used_meta":117163,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":21232,"internal_metadata":119975184},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:30.156 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:30.341 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:30.341 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:30.351 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2026-02-21T22:23:30.228588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235084,"kb_used_data":684,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508596,"statfs":{"total":193273528320,"available":193032802304,"internally_reserved":0,"allocated":700416,"data_stored":400395,"data_compressed":46044,"data_compressed_allocated":49152,"data_compressed_original":147456,"omap_allocated":41162,"internal_metadata":239951670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7746,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001176"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7300,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855219+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:29.855219+0000","last_peered":"2026-02-21T22:23:29.855219+0000","last_clean":"2026-02-21T22:23:29.855219+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:29.855219+0000","last_undegraded":"2026-02-21T22:23:29.855219+0000","last_fullsized":"2026-02-21T22:23:29.855219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":20295,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:25.352296+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:25.352296+0000","last_peered":"2026-02-21T22:23:25.352296+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:25.352296+0000","last_undegraded":"2026-02-21T22:23:25.352296+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5874,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2937},{"shards":"3","objects":4323}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7196,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855169+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:29.855169+0000","last_peered":"2026-02-21T22:23:29.855169+0000","last_clean":"2026-02-21T22:23:29.855169+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:29.855169+0000","last_undegraded":"2026-02-21T22:23:29.855169+0000","last_fullsized":"2026-02-21T22:23:29.855169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":760405,"data_compressed":191850,"data_compressed_allocated":204800,"data_compressed_original":614400,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116999,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":21,"kb_used_meta":117162,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":21883,"internal_metadata":119974533},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:30.353 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:30.353 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7370, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.367583+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:15.367583+0000', 'last_peered': '2026-02-21T22:23:15.367583+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:15.367583+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:30.353 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:30.353 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7199, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.367570+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:15.367570+0000', 'last_peered': '2026-02-21T22:23:15.367570+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:15.367570+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:30.353 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 20295, 'reported_epoch': 35, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:25.352296+0000', 'last_change': '2026-02-21T22:23:13.940568+0000', 'last_active': '2026-02-21T22:23:25.352296+0000', 'last_peered': '2026-02-21T22:23:25.352296+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:23:08.452135+0000', 'last_became_peered': '2026-02-21T22:23:08.452135+0000', 'last_unstale': '2026-02-21T22:23:25.352296+0000', 'last_undegraded': '2026-02-21T22:23:25.352296+0000', 'last_fullsized': '2026-02-21T22:23:08.351210+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 5874, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4323, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 2937}, {'shards': '3', 'objects': 4323}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7278, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363914+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:15.363914+0000', 'last_peered': '2026-02-21T22:23:15.363914+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:15.363914+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7281, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363776+0000', 'last_change': '2026-02-21T22:22:58.402976+0000', 'last_active': '2026-02-21T22:23:15.363776+0000', 'last_peered': '2026-02-21T22:23:15.363776+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:15.363776+0000', 'last_undegraded': '2026-02-21T22:22:58.342736+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7228, 'num_objects_misplaced': 7228, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:30.354 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7238, 'reported_epoch': 35, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:15.363627+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:15.363627+0000', 'last_peered': '2026-02-21T22:23:15.363627+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:15.363627+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:30.355 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:30.585 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:30.585 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34406/115430 objects degraded (29.807%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1115,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":36,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+clean","count":2},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240680960,"bytes_avail":193032847360,"bytes_total":193273528320,"degraded_objects":34406,"degraded_total":115430,"degraded_ratio":0.29806809321666811,"misplaced_objects":40280,"misplaced_total":115430,"misplaced_ratio":0.34895607727627131,"recovering_objects_per_sec":927,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":11133,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:30.596 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:30.596 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:30.785 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:30.786 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:30.796 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2026-02-21T22:23:30.228588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235084,"kb_used_data":684,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508596,"statfs":{"total":193273528320,"available":193032802304,"internally_reserved":0,"allocated":700416,"data_stored":400395,"data_compressed":46044,"data_compressed_allocated":49152,"data_compressed_original":147456,"omap_allocated":41162,"internal_metadata":239951670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7746,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001176"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7300,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855219+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:29.855219+0000","last_peered":"2026-02-21T22:23:29.855219+0000","last_clean":"2026-02-21T22:23:29.855219+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:29.855219+0000","last_undegraded":"2026-02-21T22:23:29.855219+0000","last_fullsized":"2026-02-21T22:23:29.855219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7370,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367583+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:15.367583+0000","last_peered":"2026-02-21T22:23:15.367583+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:15.367583+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7199,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.367570+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:15.367570+0000","last_peered":"2026-02-21T22:23:15.367570+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:15.367570+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":20295,"reported_epoch":35,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:25.352296+0000","last_change":"2026-02-21T22:23:13.940568+0000","last_active":"2026-02-21T22:23:25.352296+0000","last_peered":"2026-02-21T22:23:25.352296+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:23:08.452135+0000","last_became_peered":"2026-02-21T22:23:08.452135+0000","last_unstale":"2026-02-21T22:23:25.352296+0000","last_undegraded":"2026-02-21T22:23:25.352296+0000","last_fullsized":"2026-02-21T22:23:08.351210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5874,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2937},{"shards":"3","objects":4323}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7278,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363914+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:15.363914+0000","last_peered":"2026-02-21T22:23:15.363914+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:15.363914+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7281,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363776+0000","last_change":"2026-02-21T22:22:58.402976+0000","last_active":"2026-02-21T22:23:15.363776+0000","last_peered":"2026-02-21T22:23:15.363776+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:15.363776+0000","last_undegraded":"2026-02-21T22:22:58.342736+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7228,"num_objects_misplaced":7228,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7228}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7238,"reported_epoch":35,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:15.363627+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:15.363627+0000","last_peered":"2026-02-21T22:23:15.363627+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:15.363627+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7196,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855169+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:29.855169+0000","last_peered":"2026-02-21T22:23:29.855169+0000","last_clean":"2026-02-21T22:23:29.855169+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:29.855169+0000","last_undegraded":"2026-02-21T22:23:29.855169+0000","last_fullsized":"2026-02-21T22:23:29.855169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34406,"num_objects_misplaced":40280,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18722,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":760405,"data_compressed":191850,"data_compressed_allocated":204800,"data_compressed_original":614400,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":10,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986181,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964116999,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":21,"kb_used_meta":117162,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":21883,"internal_metadata":119974533},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117520,"kb_used_data":320,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254320,"statfs":{"total":96636764160,"available":96516423680,"internally_reserved":0,"allocated":327680,"data_stored":168108,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":60824,"data_compressed":15348,"data_compressed_allocated":16384,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:33.797 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:33.979 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:33.980 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:33.989 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":591,"stamp":"2026-02-21T22:23:32.228830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":41338,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14547,"num_bytes_recovered":60862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235128,"kb_used_data":728,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508552,"statfs":{"total":193273528320,"available":193032757248,"internally_reserved":0,"allocated":745472,"data_stored":464574,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41162,"internal_metadata":239951670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-3218,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2037,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000942"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7300,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855219+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:29.855219+0000","last_peered":"2026-02-21T22:23:29.855219+0000","last_clean":"2026-02-21T22:23:29.855219+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:29.855219+0000","last_undegraded":"2026-02-21T22:23:29.855219+0000","last_fullsized":"2026-02-21T22:23:29.855219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.855384+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:29.855384+0000","last_peered":"2026-02-21T22:23:29.855384+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:29.855384+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7201,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.857400+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:29.857400+0000","last_peered":"2026-02-21T22:23:29.857400+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:29.857400+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7318,"reported_epoch":36,"state":"peering","last_fresh":"2026-02-21T22:23:29.854978+0000","last_change":"2026-02-21T22:23:29.854978+0000","last_active":"2026-02-21T22:23:29.854978+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:23:29.854978+0000","last_undegraded":"2026-02-21T22:23:29.854978+0000","last_fullsized":"2026-02-21T22:23:29.854978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7280,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.855586+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:29.855586+0000","last_peered":"2026-02-21T22:23:29.855586+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:29.855586+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7728,"reported_epoch":36,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:30.285256+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:30.285256+0000","last_peered":"2026-02-21T22:23:30.285256+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:30.285256+0000","last_undegraded":"2026-02-21T22:23:30.285256+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14160,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":148},{"shards":"2,3","objects":7080}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.854782+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:29.854782+0000","last_peered":"2026-02-21T22:23:29.854782+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:29.854782+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7196,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855169+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:29.855169+0000","last_peered":"2026-02-21T22:23:29.855169+0000","last_clean":"2026-02-21T22:23:29.855169+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:29.855169+0000","last_undegraded":"2026-02-21T22:23:29.855169+0000","last_fullsized":"2026-02-21T22:23:29.855169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":41338,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14547,"num_bytes_recovered":60862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986182,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117000,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":21,"kb_used_meta":117162,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":21883,"internal_metadata":119974533},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:33.991 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:34.186 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:34.186 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:34.196 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":591,"stamp":"2026-02-21T22:23:32.228830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":41338,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14547,"num_bytes_recovered":60862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235128,"kb_used_data":728,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508552,"statfs":{"total":193273528320,"available":193032757248,"internally_reserved":0,"allocated":745472,"data_stored":464574,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41162,"internal_metadata":239951670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-3218,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2037,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000942"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7300,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855219+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:29.855219+0000","last_peered":"2026-02-21T22:23:29.855219+0000","last_clean":"2026-02-21T22:23:29.855219+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:29.855219+0000","last_undegraded":"2026-02-21T22:23:29.855219+0000","last_fullsized":"2026-02-21T22:23:29.855219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.855384+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:29.855384+0000","last_peered":"2026-02-21T22:23:29.855384+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:29.855384+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7201,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.857400+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:29.857400+0000","last_peered":"2026-02-21T22:23:29.857400+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:29.857400+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7318,"reported_epoch":36,"state":"peering","last_fresh":"2026-02-21T22:23:29.854978+0000","last_change":"2026-02-21T22:23:29.854978+0000","last_active":"2026-02-21T22:23:29.854978+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:23:29.854978+0000","last_undegraded":"2026-02-21T22:23:29.854978+0000","last_fullsized":"2026-02-21T22:23:29.854978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7280,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.855586+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:29.855586+0000","last_peered":"2026-02-21T22:23:29.855586+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:29.855586+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7728,"reported_epoch":36,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:30.285256+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:30.285256+0000","last_peered":"2026-02-21T22:23:30.285256+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:30.285256+0000","last_undegraded":"2026-02-21T22:23:30.285256+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14160,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":148},{"shards":"2,3","objects":7080}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.854782+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:29.854782+0000","last_peered":"2026-02-21T22:23:29.854782+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:29.854782+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7196,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855169+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:29.855169+0000","last_peered":"2026-02-21T22:23:29.855169+0000","last_clean":"2026-02-21T22:23:29.855169+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:29.855169+0000","last_undegraded":"2026-02-21T22:23:29.855169+0000","last_fullsized":"2026-02-21T22:23:29.855169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":41338,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14547,"num_bytes_recovered":60862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986182,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117000,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":21,"kb_used_meta":117162,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":21883,"internal_metadata":119974533},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:34.197 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:34.197 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7372, 'reported_epoch': 36, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:29.855384+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:29.855384+0000', 'last_peered': '2026-02-21T22:23:29.855384+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:29.855384+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:34.198 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:34.198 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7201, 'reported_epoch': 36, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:29.857400+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:29.857400+0000', 'last_peered': '2026-02-21T22:23:29.857400+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:29.857400+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:34.198 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-21T22:23:34.198 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'7260", 'reported_seq': 7318, 'reported_epoch': 36, 'state': 'peering', 'last_fresh': '2026-02-21T22:23:29.854978+0000', 'last_change': '2026-02-21T22:23:29.854978+0000', 'last_active': '2026-02-21T22:23:29.854978+0000', 'last_peered': '2026-02-21T22:22:21.565172+0000', 'last_clean': '2026-02-21T22:22:21.565172+0000', 'last_became_active': '2026-02-21T22:05:13.219955+0000', 'last_became_peered': '2026-02-21T22:05:13.219955+0000', 'last_unstale': '2026-02-21T22:23:29.854978+0000', 'last_undegraded': '2026-02-21T22:23:29.854978+0000', 'last_fullsized': '2026-02-21T22:23:29.854978+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7260, 'log_dups_size': 0, 'ondisk_log_size': 7260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000232905, 'stat_sum': {'num_bytes': 68080, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 7316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2026-02-21T22:23:34.198 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:34.198 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7280, 'reported_epoch': 36, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:29.855586+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:29.855586+0000', 'last_peered': '2026-02-21T22:23:29.855586+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:29.855586+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:34.199 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:34.199 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 7728, 'reported_epoch': 36, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:30.285256+0000', 'last_change': '2026-02-21T22:23:29.806838+0000', 'last_active': '2026-02-21T22:23:30.285256+0000', 'last_peered': '2026-02-21T22:23:30.285256+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:30.285256+0000', 'last_undegraded': '2026-02-21T22:23:30.285256+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 14160, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 149, 'num_bytes_recovered': 38, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 148}, {'shards': '2,3', 'objects': 7080}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:34.199 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:34.199 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7240, 'reported_epoch': 36, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:29.854782+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:29.854782+0000', 'last_peered': '2026-02-21T22:23:29.854782+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:29.854782+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:34.199 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:34.429 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:34.429 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27178/115430 objects degraded (23.545%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1119,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":37,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+clean","count":2},{"state_name":"active+recovering+undersized+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240771072,"bytes_avail":193032757248,"bytes_total":193273528320,"inactive_pgs_ratio":0.125,"degraded_objects":27178,"degraded_total":115430,"degraded_ratio":0.23545005631118424,"misplaced_objects":41338,"misplaced_total":115430,"misplaced_ratio":0.35812180542320021,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":2,"recovering_keys_per_sec":0,"num_objects_recovered":0,"num_bytes_recovered":19,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:34.438 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:34.439 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:34.616 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:34.616 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:34.627 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":592,"stamp":"2026-02-21T22:23:34.229069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":41338,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14547,"num_bytes_recovered":60862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235128,"kb_used_data":728,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508552,"statfs":{"total":193273528320,"available":193032757248,"internally_reserved":0,"allocated":745472,"data_stored":464574,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41162,"internal_metadata":239951670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":1058,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-4175,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000937"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7300,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855219+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:29.855219+0000","last_peered":"2026-02-21T22:23:29.855219+0000","last_clean":"2026-02-21T22:23:29.855219+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:29.855219+0000","last_undegraded":"2026-02-21T22:23:29.855219+0000","last_fullsized":"2026-02-21T22:23:29.855219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.855384+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:29.855384+0000","last_peered":"2026-02-21T22:23:29.855384+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:29.855384+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7201,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.857400+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:29.857400+0000","last_peered":"2026-02-21T22:23:29.857400+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:29.857400+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7318,"reported_epoch":36,"state":"peering","last_fresh":"2026-02-21T22:23:29.854978+0000","last_change":"2026-02-21T22:23:29.854978+0000","last_active":"2026-02-21T22:23:29.854978+0000","last_peered":"2026-02-21T22:22:21.565172+0000","last_clean":"2026-02-21T22:22:21.565172+0000","last_became_active":"2026-02-21T22:05:13.219955+0000","last_became_peered":"2026-02-21T22:05:13.219955+0000","last_unstale":"2026-02-21T22:23:29.854978+0000","last_undegraded":"2026-02-21T22:23:29.854978+0000","last_fullsized":"2026-02-21T22:23:29.854978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7280,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.855586+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:29.855586+0000","last_peered":"2026-02-21T22:23:29.855586+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:29.855586+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7728,"reported_epoch":36,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:30.285256+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:30.285256+0000","last_peered":"2026-02-21T22:23:30.285256+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:30.285256+0000","last_undegraded":"2026-02-21T22:23:30.285256+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14160,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":148},{"shards":"2,3","objects":7080}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":36,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:29.854782+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:29.854782+0000","last_peered":"2026-02-21T22:23:29.854782+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:29.854782+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7196,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-21T22:23:29.855169+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:29.855169+0000","last_peered":"2026-02-21T22:23:29.855169+0000","last_clean":"2026-02-21T22:23:29.855169+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:29.855169+0000","last_undegraded":"2026-02-21T22:23:29.855169+0000","last_fullsized":"2026-02-21T22:23:29.855169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":41338,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14547,"num_bytes_recovered":60862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986182,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117000,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":21,"kb_used_meta":117162,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":21883,"internal_metadata":119974533},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117564,"kb_used_data":364,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254276,"statfs":{"total":96636764160,"available":96516378624,"internally_reserved":0,"allocated":372736,"data_stored":232287,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:37.629 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:37.806 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:37.807 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:37.817 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":593,"stamp":"2026-02-21T22:23:36.229335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-2306,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000969"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":12776,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:35.286046+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:35.286046+0000","last_peered":"2026-02-21T22:23:35.286046+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:35.286046+0000","last_undegraded":"2026-02-21T22:23:35.286046+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10796,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1831,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1830},{"shards":"2,3","objects":5398}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986183,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117001,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:37.818 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:37.998 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:37.998 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:38.008 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":593,"stamp":"2026-02-21T22:23:36.229335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-2306,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000969"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":12776,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:35.286046+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:35.286046+0000","last_peered":"2026-02-21T22:23:35.286046+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:35.286046+0000","last_undegraded":"2026-02-21T22:23:35.286046+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10796,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1831,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1830},{"shards":"2,3","objects":5398}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986183,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117001,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:38.009 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:38.009 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7374, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857529+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:30.857529+0000', 'last_peered': '2026-02-21T22:23:30.857529+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:30.857529+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7203, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.861178+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:30.861178+0000', 'last_peered': '2026-02-21T22:23:30.861178+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:30.861178+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7282, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857792+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:30.857792+0000', 'last_peered': '2026-02-21T22:23:30.857792+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:30.857792+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 12776, 'reported_epoch': 37, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:35.286046+0000', 'last_change': '2026-02-21T22:23:29.806838+0000', 'last_active': '2026-02-21T22:23:35.286046+0000', 'last_peered': '2026-02-21T22:23:35.286046+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:35.286046+0000', 'last_undegraded': '2026-02-21T22:23:35.286046+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 10796, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1831, 'num_bytes_recovered': 38, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 1830}, {'shards': '2,3', 'objects': 5398}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:38.010 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7242, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857265+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:30.857265+0000', 'last_peered': '2026-02-21T22:23:30.857265+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:30.857265+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:38.011 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:38.231 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:38.231 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27178/115430 objects degraded (23.545%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1123,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":37,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+clean","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240787456,"bytes_avail":193032740864,"bytes_total":193273528320,"degraded_objects":27178,"degraded_total":115430,"degraded_ratio":0.23545005631118424,"misplaced_objects":37974,"misplaced_total":115430,"misplaced_ratio":0.32897860174997828,"recovering_objects_per_sec":595,"recovering_bytes_per_sec":7604,"recovering_keys_per_sec":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:38.242 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:38.243 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:38.427 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:38.427 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:38.438 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":594,"stamp":"2026-02-21T22:23:38.229575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-2306,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.375323"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":12776,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:35.286046+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:35.286046+0000","last_peered":"2026-02-21T22:23:35.286046+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:35.286046+0000","last_undegraded":"2026-02-21T22:23:35.286046+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10796,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1831,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1830},{"shards":"2,3","objects":5398}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986183,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117001,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:41.439 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:41.624 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:41.624 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:41.633 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2026-02-21T22:23:40.229814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-2306,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001226"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":12776,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:35.286046+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:35.286046+0000","last_peered":"2026-02-21T22:23:35.286046+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:35.286046+0000","last_undegraded":"2026-02-21T22:23:35.286046+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10796,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1831,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1830},{"shards":"2,3","objects":5398}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986183,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117001,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:41.634 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:41.822 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:41.823 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:41.834 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2026-02-21T22:23:40.229814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7228,"num_objects_misplaced":-2306,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001226"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":12776,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:35.286046+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:35.286046+0000","last_peered":"2026-02-21T22:23:35.286046+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:35.286046+0000","last_undegraded":"2026-02-21T22:23:35.286046+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10796,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1831,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1830},{"shards":"2,3","objects":5398}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":37974,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23489,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986183,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117001,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:41.835 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:41.836 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7374, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857529+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:30.857529+0000', 'last_peered': '2026-02-21T22:23:30.857529+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:30.857529+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:41.836 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:41.836 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7203, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.861178+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:30.861178+0000', 'last_peered': '2026-02-21T22:23:30.861178+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:30.861178+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:41.836 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:41.836 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7282, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857792+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:30.857792+0000', 'last_peered': '2026-02-21T22:23:30.857792+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:30.857792+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:41.836 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:41.837 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 12776, 'reported_epoch': 37, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:35.286046+0000', 'last_change': '2026-02-21T22:23:29.806838+0000', 'last_active': '2026-02-21T22:23:35.286046+0000', 'last_peered': '2026-02-21T22:23:35.286046+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:35.286046+0000', 'last_undegraded': '2026-02-21T22:23:35.286046+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 10796, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1831, 'num_bytes_recovered': 38, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 1830}, {'shards': '2,3', 'objects': 5398}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:41.837 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:41.837 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7242, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857265+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:30.857265+0000', 'last_peered': '2026-02-21T22:23:30.857265+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:30.857265+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:41.837 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:42.063 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:42.063 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27178/115430 objects degraded (23.545%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1126,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":37,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+clean","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240787456,"bytes_avail":193032740864,"bytes_total":193273528320,"degraded_objects":27178,"degraded_total":115430,"degraded_ratio":0.23545005631118424,"misplaced_objects":37974,"misplaced_total":115430,"misplaced_ratio":0.32897860174997828,"recovering_objects_per_sec":476,"recovering_bytes_per_sec":6083,"recovering_keys_per_sec":0,"num_objects_recovered":4767,"num_bytes_recovered":60843,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:42.073 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:42.073 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:42.263 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:42.263 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:42.275 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2026-02-21T22:23:42.230040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7404,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10962,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.373311"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":18836,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:40.286607+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:40.286607+0000","last_peered":"2026-02-21T22:23:40.286607+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:40.286607+0000","last_undegraded":"2026-02-21T22:23:40.286607+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6756,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3851,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3850},{"shards":"2,3","objects":3378}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986184,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117002,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:45.277 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:45.468 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:45.469 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:45.479 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2026-02-21T22:23:44.230300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7404,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10962,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":18836,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:40.286607+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:40.286607+0000","last_peered":"2026-02-21T22:23:40.286607+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:40.286607+0000","last_undegraded":"2026-02-21T22:23:40.286607+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6756,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3851,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3850},{"shards":"2,3","objects":3378}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986184,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117002,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:45.481 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:45.664 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:45.664 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:45.674 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2026-02-21T22:23:44.230300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7404,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10962,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":18836,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:40.286607+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:40.286607+0000","last_peered":"2026-02-21T22:23:40.286607+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:40.286607+0000","last_undegraded":"2026-02-21T22:23:40.286607+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6756,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3851,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3850},{"shards":"2,3","objects":3378}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986184,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117002,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:45.675 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:45.675 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7374, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857529+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:30.857529+0000', 'last_peered': '2026-02-21T22:23:30.857529+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:30.857529+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:45.676 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:45.676 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7203, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.861178+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:30.861178+0000', 'last_peered': '2026-02-21T22:23:30.861178+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:30.861178+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:45.676 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:45.676 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7282, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857792+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:30.857792+0000', 'last_peered': '2026-02-21T22:23:30.857792+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:30.857792+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:45.676 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:45.676 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 18836, 'reported_epoch': 37, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:40.286607+0000', 'last_change': '2026-02-21T22:23:29.806838+0000', 'last_active': '2026-02-21T22:23:40.286607+0000', 'last_peered': '2026-02-21T22:23:40.286607+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:40.286607+0000', 'last_undegraded': '2026-02-21T22:23:40.286607+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 6756, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3851, 'num_bytes_recovered': 38, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 3850}, {'shards': '2,3', 'objects': 3378}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:45.677 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:45.677 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7242, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857265+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:30.857265+0000', 'last_peered': '2026-02-21T22:23:30.857265+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:30.857265+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:45.677 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:45.906 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:45.906 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27178/115430 objects degraded (23.545%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1130,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":37,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+clean","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":240787456,"bytes_avail":193032740864,"bytes_total":193273528320,"degraded_objects":27178,"degraded_total":115430,"degraded_ratio":0.23545005631118424,"misplaced_objects":33934,"misplaced_total":115430,"misplaced_ratio":0.29397903491293426,"recovering_objects_per_sec":913,"recovering_bytes_per_sec":5068,"recovering_keys_per_sec":0,"num_objects_recovered":10962,"num_bytes_recovered":60824,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:45.916 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:45.916 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:46.101 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:46.102 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:46.111 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2026-02-21T22:23:44.230300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":235144,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":234327,"kb_avail":188508536,"statfs":{"total":193273528320,"available":193032740864,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":239951019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7404,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10962,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":18836,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:40.286607+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:40.286607+0000","last_peered":"2026-02-21T22:23:40.286607+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:40.286607+0000","last_undegraded":"2026-02-21T22:23:40.286607+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6756,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3851,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3850},{"shards":"2,3","objects":3378}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":33934,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25509,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986184,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117002,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":117161,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":119973882},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117572,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":117165,"kb_avail":94254268,"statfs":{"total":96636764160,"available":96516370432,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":119977137},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:49.113 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:49.291 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:49.292 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:49.301 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2026-02-21T22:23:48.230821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":27822,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28565,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":289288,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":288471,"kb_avail":188454392,"statfs":{"total":193273528320,"available":192977297408,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":295394475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10152,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5076,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001006"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":28004,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:45.287187+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:45.287187+0000","last_peered":"2026-02-21T22:23:45.287187+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:45.287187+0000","last_undegraded":"2026-02-21T22:23:45.287187+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":644,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6907,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6906},{"shards":"2,3","objects":322}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":27822,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28565,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986185,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117003,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135620,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":135209,"kb_avail":94236220,"statfs":{"total":96636764160,"available":96497889280,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":138455034},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153668,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218172,"statfs":{"total":96636764160,"available":96479408128,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:49.303 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:49.482 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:49.483 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:49.492 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2026-02-21T22:23:48.230821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":27822,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28565,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":289288,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":288471,"kb_avail":188454392,"statfs":{"total":193273528320,"available":192977297408,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":295394475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10152,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5076,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001006"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":28004,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:45.287187+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:45.287187+0000","last_peered":"2026-02-21T22:23:45.287187+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:45.287187+0000","last_undegraded":"2026-02-21T22:23:45.287187+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":644,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6907,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6906},{"shards":"2,3","objects":322}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":27822,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28565,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986185,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117003,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135620,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":135209,"kb_avail":94236220,"statfs":{"total":96636764160,"available":96497889280,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":138455034},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153668,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218172,"statfs":{"total":96636764160,"available":96479408128,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:49.493 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:49.493 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7374, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857529+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:30.857529+0000', 'last_peered': '2026-02-21T22:23:30.857529+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:30.857529+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7203, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.861178+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:30.861178+0000', 'last_peered': '2026-02-21T22:23:30.861178+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:30.861178+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7282, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857792+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:30.857792+0000', 'last_peered': '2026-02-21T22:23:30.857792+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:30.857792+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'7229", 'reported_seq': 28004, 'reported_epoch': 37, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:45.287187+0000', 'last_change': '2026-02-21T22:23:29.806838+0000', 'last_active': '2026-02-21T22:23:45.287187+0000', 'last_peered': '2026-02-21T22:23:45.287187+0000', 'last_clean': '2026-02-21T22:22:21.407133+0000', 'last_became_active': '2026-02-21T22:22:58.395667+0000', 'last_became_peered': '2026-02-21T22:22:58.395667+0000', 'last_unstale': '2026-02-21T22:23:45.287187+0000', 'last_undegraded': '2026-02-21T22:23:45.287187+0000', 'last_fullsized': '2026-02-21T22:22:58.288244+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7229, 'log_dups_size': 0, 'ondisk_log_size': 7229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000254305, 'stat_sum': {'num_bytes': 37656, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 644, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 7257, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6907, 'num_bytes_recovered': 38, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 6906}, {'shards': '2,3', 'objects': 322}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:49.494 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:49.495 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 7242, 'reported_epoch': 37, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:30.857265+0000', 'last_change': '2026-02-21T22:23:08.478599+0000', 'last_active': '2026-02-21T22:23:30.857265+0000', 'last_peered': '2026-02-21T22:23:30.857265+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:30.857265+0000', 'last_undegraded': '2026-02-21T22:23:08.412133+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7173, 'num_objects_misplaced': 7173, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 7173}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:49.495 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:49.715 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:49.715 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27178/115430 objects degraded (23.545%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1134,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+clean","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":296230912,"bytes_avail":192977297408,"bytes_total":193273528320,"degraded_objects":27178,"degraded_total":115430,"degraded_ratio":0.23545005631118424,"misplaced_objects":27822,"misplaced_total":115430,"misplaced_ratio":0.24102919518322793,"recovering_objects_per_sec":634,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":5076,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:49.725 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:49.725 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:49.908 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:49.909 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:49.918 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2026-02-21T22:23:48.230821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":27822,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28565,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":289288,"kb_used_data":744,"kb_used_omap":40,"kb_used_meta":288471,"kb_avail":188454392,"statfs":{"total":193273528320,"available":192977297408,"internally_reserved":0,"allocated":761856,"data_stored":471204,"data_compressed":61392,"data_compressed_allocated":65536,"data_compressed_original":196608,"omap_allocated":41813,"internal_metadata":295394475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10152,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5076,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001006"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7302,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:30.857543+0000","last_peered":"2026-02-21T22:23:30.857543+0000","last_clean":"2026-02-21T22:23:30.857543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:30.857543+0000","last_undegraded":"2026-02-21T22:23:30.857543+0000","last_fullsized":"2026-02-21T22:23:30.857543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7374,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857529+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:30.857529+0000","last_peered":"2026-02-21T22:23:30.857529+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:30.857529+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7203,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.861178+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:30.861178+0000","last_peered":"2026-02-21T22:23:30.861178+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:30.861178+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7327,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.861893+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:30.861893+0000","last_peered":"2026-02-21T22:23:30.861893+0000","last_clean":"2026-02-21T22:23:30.861893+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:30.861893+0000","last_undegraded":"2026-02-21T22:23:30.861893+0000","last_fullsized":"2026-02-21T22:23:30.861893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7282,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857792+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:30.857792+0000","last_peered":"2026-02-21T22:23:30.857792+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:30.857792+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":28004,"reported_epoch":37,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:45.287187+0000","last_change":"2026-02-21T22:23:29.806838+0000","last_active":"2026-02-21T22:23:45.287187+0000","last_peered":"2026-02-21T22:23:45.287187+0000","last_clean":"2026-02-21T22:22:21.407133+0000","last_became_active":"2026-02-21T22:22:58.395667+0000","last_became_peered":"2026-02-21T22:22:58.395667+0000","last_unstale":"2026-02-21T22:23:45.287187+0000","last_undegraded":"2026-02-21T22:23:45.287187+0000","last_fullsized":"2026-02-21T22:22:58.288244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":644,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6907,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6906},{"shards":"2,3","objects":322}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7242,"reported_epoch":37,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:30.857265+0000","last_change":"2026-02-21T22:23:08.478599+0000","last_active":"2026-02-21T22:23:30.857265+0000","last_peered":"2026-02-21T22:23:30.857265+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:30.857265+0000","last_undegraded":"2026-02-21T22:23:08.412133+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7173,"num_objects_misplaced":7173,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":7173}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7198,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-21T22:23:30.857505+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:30.857505+0000","last_peered":"2026-02-21T22:23:30.857505+0000","last_clean":"2026-02-21T22:23:30.857505+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:30.857505+0000","last_undegraded":"2026-02-21T22:23:30.857505+0000","last_fullsized":"2026-02-21T22:23:30.857505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27178,"num_objects_misplaced":27822,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28565,"num_bytes_recovered":121686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":821248,"data_compressed":207198,"data_compressed_allocated":221184,"data_compressed_original":663552,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":11,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986185,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117003,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135620,"kb_used_data":372,"kb_used_omap":22,"kb_used_meta":135209,"kb_avail":94236220,"statfs":{"total":96636764160,"available":96497889280,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":22534,"internal_metadata":138455034},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153668,"kb_used_data":372,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218172,"statfs":{"total":96636764160,"available":96479408128,"internally_reserved":0,"allocated":380928,"data_stored":235602,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":121667,"data_compressed":30696,"data_compressed_allocated":32768,"data_compressed_original":98304,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:52.920 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:53.101 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:53.101 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:53.111 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":603,"stamp":"2026-02-21T22:23:52.231329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":31231,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30447,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-2703,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4938,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001029"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":11927,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:50.353690+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:50.353690+0000","last_peered":"2026-02-21T22:23:50.353690+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:50.353690+0000","last_undegraded":"2026-02-21T22:23:50.353690+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11226,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1560,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5613},{"shards":"3","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":31231,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30447,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986186,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117004,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:53.112 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:53.297 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:53.298 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:53.308 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":603,"stamp":"2026-02-21T22:23:52.231329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":31231,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30447,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-2703,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4938,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001029"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":11927,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:50.353690+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:50.353690+0000","last_peered":"2026-02-21T22:23:50.353690+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:50.353690+0000","last_undegraded":"2026-02-21T22:23:50.353690+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11226,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1560,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5613},{"shards":"3","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":31231,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30447,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986186,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117004,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:53.309 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:53.309 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7378, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296928+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:47.296928+0000', 'last_peered': '2026-02-21T22:23:47.296928+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:47.296928+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:53.310 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:53.310 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7207, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296726+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:47.296726+0000', 'last_peered': '2026-02-21T22:23:47.296726+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:47.296726+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:53.310 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:53.310 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7286, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296714+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:47.296714+0000', 'last_peered': '2026-02-21T22:23:47.296714+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:47.296714+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:53.310 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:53.310 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 11927, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:50.353690+0000', 'last_change': '2026-02-21T22:23:45.657565+0000', 'last_active': '2026-02-21T22:23:50.353690+0000', 'last_peered': '2026-02-21T22:23:50.353690+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:50.353690+0000', 'last_undegraded': '2026-02-21T22:23:50.353690+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 11226, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1560, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5613}, {'shards': '3', 'objects': 1560}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:53.311 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:53.539 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:53.539 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20005/115430 objects degraded (17.331%), 3 pgs degraded","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1138,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":314777600,"bytes_avail":192958750720,"bytes_total":193273528320,"degraded_objects":20005,"degraded_total":115430,"degraded_ratio":0.17330849865719483,"misplaced_objects":31231,"misplaced_total":115430,"misplaced_ratio":0.27056224551676339,"recovering_objects_per_sec":617,"recovering_bytes_per_sec":3801,"recovering_keys_per_sec":0,"num_objects_recovered":4938,"num_bytes_recovered":30412,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-21T22:21:00.210624+0000","services":{}},"progress_events":{}} 2026-02-21T22:23:53.550 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:53.550 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:53.735 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:53.736 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:53.745 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":603,"stamp":"2026-02-21T22:23:52.231329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":31231,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30447,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-2703,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4938,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001029"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":11927,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:50.353690+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:50.353690+0000","last_peered":"2026-02-21T22:23:50.353690+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:50.353690+0000","last_undegraded":"2026-02-21T22:23:50.353690+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11226,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1560,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5613},{"shards":"3","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":31231,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30447,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986186,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117004,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:56.747 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:56.926 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:56.926 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:56.936 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":605,"stamp":"2026-02-21T22:23:56.231805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-573,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.936718"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":17900,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:55.355025+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:55.355025+0000","last_peered":"2026-02-21T22:23:55.355025+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:55.355025+0000","last_undegraded":"2026-02-21T22:23:55.355025+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7244,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":3622},{"shards":"3","objects":3551}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986187,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117005,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:56.938 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:57.123 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:57.123 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:57.133 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":605,"stamp":"2026-02-21T22:23:56.231805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-573,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.936718"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":17900,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:55.355025+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:55.355025+0000","last_peered":"2026-02-21T22:23:55.355025+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:55.355025+0000","last_undegraded":"2026-02-21T22:23:55.355025+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7244,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":3622},{"shards":"3","objects":3551}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986187,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117005,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:23:57.134 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:23:57.134 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7378, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296928+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:47.296928+0000', 'last_peered': '2026-02-21T22:23:47.296928+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:47.296928+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:57.134 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:23:57.135 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7207, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296726+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:47.296726+0000', 'last_peered': '2026-02-21T22:23:47.296726+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:47.296726+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:57.135 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:23:57.135 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7286, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296714+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:47.296714+0000', 'last_peered': '2026-02-21T22:23:47.296714+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:47.296714+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:23:57.135 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:23:57.135 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 17900, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:55.355025+0000', 'last_change': '2026-02-21T22:23:45.657565+0000', 'last_active': '2026-02-21T22:23:55.355025+0000', 'last_peered': '2026-02-21T22:23:55.355025+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:55.355025+0000', 'last_undegraded': '2026-02-21T22:23:55.355025+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 7244, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3551, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 3622}, {'shards': '3', 'objects': 3551}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:23:57.135 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:23:57.360 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:57.360 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20005/115430 objects degraded (17.331%), 3 pgs degraded","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1142,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":314777600,"bytes_avail":192958750720,"bytes_total":193273528320,"degraded_objects":20005,"degraded_total":115430,"degraded_ratio":0.17330849865719483,"misplaced_objects":27249,"misplaced_total":115430,"misplaced_ratio":0.23606514770856793,"recovering_objects_per_sec":389,"recovering_bytes_per_sec":3060,"recovering_keys_per_sec":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":6,"modified":"2026-02-21T22:23:56.231851+0000","services":{"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-21T22:23:57.370 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:23:57.370 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:23:57.556 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:23:57.556 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:23:57.565 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":605,"stamp":"2026-02-21T22:23:56.231805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-573,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.936718"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":17900,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:55.355025+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:55.355025+0000","last_peered":"2026-02-21T22:23:55.355025+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:55.355025+0000","last_undegraded":"2026-02-21T22:23:55.355025+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7244,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":3622},{"shards":"3","objects":3551}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986187,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117005,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:00.568 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:00.753 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:00.754 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:00.763 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2026-02-21T22:24:00.232271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-573,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":17900,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:55.355025+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:55.355025+0000","last_peered":"2026-02-21T22:23:55.355025+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:55.355025+0000","last_undegraded":"2026-02-21T22:23:55.355025+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7244,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":3622},{"shards":"3","objects":3551}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986187,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117005,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:00.764 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:00.959 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:00.960 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:00.970 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2026-02-21T22:24:00.232271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-573,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":17900,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:55.355025+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:55.355025+0000","last_peered":"2026-02-21T22:23:55.355025+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:55.355025+0000","last_undegraded":"2026-02-21T22:23:55.355025+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7244,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":3622},{"shards":"3","objects":3551}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986187,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117005,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:00.971 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:00.971 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7378, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296928+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:47.296928+0000', 'last_peered': '2026-02-21T22:23:47.296928+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:47.296928+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:00.971 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:00.972 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7207, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296726+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:47.296726+0000', 'last_peered': '2026-02-21T22:23:47.296726+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:47.296726+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:00.972 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:24:00.972 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7286, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296714+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:47.296714+0000', 'last_peered': '2026-02-21T22:23:47.296714+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:47.296714+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:00.972 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:24:00.972 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 17900, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:23:55.355025+0000', 'last_change': '2026-02-21T22:23:45.657565+0000', 'last_active': '2026-02-21T22:23:55.355025+0000', 'last_peered': '2026-02-21T22:23:55.355025+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:23:55.355025+0000', 'last_undegraded': '2026-02-21T22:23:55.355025+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 7244, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3551, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 3622}, {'shards': '3', 'objects': 3551}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:00.972 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:01.206 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:01.206 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20005/115430 objects degraded (17.331%), 3 pgs degraded, 2 pgs undersized","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1146,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":314777600,"bytes_avail":192958750720,"bytes_total":193273528320,"degraded_objects":20005,"degraded_total":115430,"degraded_ratio":0.17330849865719483,"misplaced_objects":27249,"misplaced_total":115430,"misplaced_ratio":0.23606514770856793,"recovering_objects_per_sec":322,"recovering_bytes_per_sec":2534,"recovering_keys_per_sec":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":6,"modified":"2026-02-21T22:23:56.231851+0000","services":{"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-21T22:24:01.217 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:01.217 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:01.396 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:01.397 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:01.406 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2026-02-21T22:24:00.232271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7173,"num_objects_misplaced":-573,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":17900,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:23:55.355025+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:23:55.355025+0000","last_peered":"2026-02-21T22:23:55.355025+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:23:55.355025+0000","last_undegraded":"2026-02-21T22:23:55.355025+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7244,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":3622},{"shards":"3","objects":3551}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":27249,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32438,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986187,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117005,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:04.408 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:04.602 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:04.602 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:04.613 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2026-02-21T22:24:04.232738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":21469,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35328,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000932"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":26570,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:00.356251+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:24:00.356251+0000","last_peered":"2026-02-21T22:24:00.356251+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:24:00.356251+0000","last_undegraded":"2026-02-21T22:24:00.356251+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1464,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6441,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":732},{"shards":"3","objects":6441}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":21469,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35328,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986188,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117006,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:04.614 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:04.791 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:04.791 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:04.801 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2026-02-21T22:24:04.232738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":21469,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35328,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000932"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":26570,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:00.356251+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:24:00.356251+0000","last_peered":"2026-02-21T22:24:00.356251+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:24:00.356251+0000","last_undegraded":"2026-02-21T22:24:00.356251+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1464,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6441,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":732},{"shards":"3","objects":6441}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":21469,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35328,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986188,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117006,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7378, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296928+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:23:47.296928+0000', 'last_peered': '2026-02-21T22:23:47.296928+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:23:47.296928+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7207, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296726+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:23:47.296726+0000', 'last_peered': '2026-02-21T22:23:47.296726+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:23:47.296726+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 7286, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:23:47.296714+0000', 'last_change': '2026-02-21T22:22:58.403452+0000', 'last_active': '2026-02-21T22:23:47.296714+0000', 'last_peered': '2026-02-21T22:23:47.296714+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:23:47.296714+0000', 'last_undegraded': '2026-02-21T22:22:58.391276+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7220, 'num_objects_misplaced': 7220, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-21T22:24:04.802 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'7173", 'reported_seq': 26570, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:00.356251+0000', 'last_change': '2026-02-21T22:23:45.657565+0000', 'last_active': '2026-02-21T22:24:00.356251+0000', 'last_peered': '2026-02-21T22:24:00.356251+0000', 'last_clean': '2026-02-21T22:22:20.769088+0000', 'last_became_active': '2026-02-21T22:23:08.475124+0000', 'last_became_peered': '2026-02-21T22:23:08.475124+0000', 'last_unstale': '2026-02-21T22:24:00.356251+0000', 'last_undegraded': '2026-02-21T22:24:00.356251+0000', 'last_fullsized': '2026-02-21T22:23:08.351171+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000216715, 'stat_sum': {'num_bytes': 22378, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1464, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 7187, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6441, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 732}, {'shards': '3', 'objects': 6441}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:04.802 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:05.027 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:05.028 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20005/115430 objects degraded (17.331%), 3 pgs degraded, 2 pgs undersized","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1149,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":41,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":314777600,"bytes_avail":192958750720,"bytes_total":193273528320,"degraded_objects":20005,"degraded_total":115430,"degraded_ratio":0.17330849865719483,"misplaced_objects":21469,"misplaced_total":115430,"misplaced_ratio":0.18599151000606429,"recovering_objects_per_sec":361,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2890,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":6,"modified":"2026-02-21T22:23:56.231851+0000","services":{"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-21T22:24:05.039 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:05.040 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:05.244 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:05.244 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:05.255 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2026-02-21T22:24:04.232738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":21469,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35328,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307400,"kb_used_data":808,"kb_used_omap":42,"kb_used_meta":306517,"kb_avail":188436280,"statfs":{"total":193273528320,"available":192958750720,"internally_reserved":0,"allocated":827392,"data_stored":545230,"data_compressed":76740,"data_compressed_allocated":81920,"data_compressed_original":245760,"omap_allocated":43115,"internal_metadata":313874325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000932"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7306,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296543+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:23:47.296543+0000","last_peered":"2026-02-21T22:23:47.296543+0000","last_clean":"2026-02-21T22:23:47.296543+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:23:47.296543+0000","last_undegraded":"2026-02-21T22:23:47.296543+0000","last_fullsized":"2026-02-21T22:23:47.296543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7378,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296928+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:23:47.296928+0000","last_peered":"2026-02-21T22:23:47.296928+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:23:47.296928+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7207,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296726+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:23:47.296726+0000","last_peered":"2026-02-21T22:23:47.296726+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:23:47.296726+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7331,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.297282+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:23:47.297282+0000","last_peered":"2026-02-21T22:23:47.297282+0000","last_clean":"2026-02-21T22:23:47.297282+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:23:47.297282+0000","last_undegraded":"2026-02-21T22:23:47.297282+0000","last_fullsized":"2026-02-21T22:23:47.297282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7286,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:23:47.296714+0000","last_change":"2026-02-21T22:22:58.403452+0000","last_active":"2026-02-21T22:23:47.296714+0000","last_peered":"2026-02-21T22:23:47.296714+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:23:47.296714+0000","last_undegraded":"2026-02-21T22:22:58.391276+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7220,"num_objects_misplaced":7220,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7220}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.300423+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:23:47.300423+0000","last_peered":"2026-02-21T22:23:47.300423+0000","last_clean":"2026-02-21T22:23:47.300423+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:23:47.300423+0000","last_undegraded":"2026-02-21T22:23:47.300423+0000","last_fullsized":"2026-02-21T22:23:47.300423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":26570,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:00.356251+0000","last_change":"2026-02-21T22:23:45.657565+0000","last_active":"2026-02-21T22:24:00.356251+0000","last_peered":"2026-02-21T22:24:00.356251+0000","last_clean":"2026-02-21T22:22:20.769088+0000","last_became_active":"2026-02-21T22:23:08.475124+0000","last_became_peered":"2026-02-21T22:23:08.475124+0000","last_unstale":"2026-02-21T22:24:00.356251+0000","last_undegraded":"2026-02-21T22:24:00.356251+0000","last_fullsized":"2026-02-21T22:23:08.351171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1464,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6441,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":732},{"shards":"3","objects":6441}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7202,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-21T22:23:47.296451+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:23:47.296451+0000","last_peered":"2026-02-21T22:23:47.296451+0000","last_clean":"2026-02-21T22:23:47.296451+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:23:47.296451+0000","last_undegraded":"2026-02-21T22:23:47.296451+0000","last_fullsized":"2026-02-21T22:23:47.296451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20005,"num_objects_misplaced":21469,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35328,"num_bytes_recovered":152098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":882072,"data_compressed":222546,"data_compressed_allocated":237568,"data_compressed_original":712704,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":12,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986188,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117006,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":23,"kb_used_meta":153256,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":23836,"internal_metadata":156934884},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153700,"kb_used_data":404,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218140,"statfs":{"total":96636764160,"available":96479375360,"internally_reserved":0,"allocated":413696,"data_stored":272615,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":152079,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:08.258 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:08.440 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:08.440 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:08.451 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":613,"stamp":"2026-02-21T22:24:08.233244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":24509,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307448,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306516,"kb_avail":188436232,"statfs":{"total":193273528320,"available":192958701568,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":313873023},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-2740,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4980,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000973"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":11365,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:05.288100+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:05.288100+0000","last_peered":"2026-02-21T22:24:05.288100+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:05.288100+0000","last_undegraded":"2026-02-21T22:24:05.288100+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11724,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1358},{"shards":"2,3","objects":5862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":24509,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986189,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117007,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:08.452 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:08.647 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:08.648 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:08.658 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":613,"stamp":"2026-02-21T22:24:08.233244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":24509,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307448,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306516,"kb_avail":188436232,"statfs":{"total":193273528320,"available":192958701568,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":313873023},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-2740,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4980,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000973"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":11365,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:05.288100+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:05.288100+0000","last_peered":"2026-02-21T22:24:05.288100+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:05.288100+0000","last_undegraded":"2026-02-21T22:24:05.288100+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11724,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1358},{"shards":"2,3","objects":5862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":24509,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986189,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117007,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:08.659 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:08.659 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7382, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426516+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:24:02.426516+0000', 'last_peered': '2026-02-21T22:24:02.426516+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:02.426516+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:08.660 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:08.660 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7211, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426252+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:02.426252+0000', 'last_peered': '2026-02-21T22:24:02.426252+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:02.426252+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:08.660 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:24:08.660 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 11365, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:05.288100+0000', 'last_change': '2026-02-21T22:24:01.205860+0000', 'last_active': '2026-02-21T22:24:05.288100+0000', 'last_peered': '2026-02-21T22:24:05.288100+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:24:05.288100+0000', 'last_undegraded': '2026-02-21T22:24:05.288100+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 11724, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1359, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 1358}, {'shards': '2,3', 'objects': 5862}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:08.660 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:08.891 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:08.892 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12785/115430 objects degraded (11.076%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1153,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":41,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":314826752,"bytes_avail":192958701568,"bytes_total":193273528320,"degraded_objects":12785,"degraded_total":115430,"degraded_ratio":0.11075976782465563,"misplaced_objects":24509,"misplaced_total":115430,"misplaced_ratio":0.21232781772502815,"recovering_objects_per_sec":622,"recovering_bytes_per_sec":1900,"recovering_keys_per_sec":0,"num_objects_recovered":4980,"num_bytes_recovered":15206,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:08.902 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:08.903 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:09.086 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:09.086 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:09.097 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":613,"stamp":"2026-02-21T22:24:08.233244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":24509,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307448,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306516,"kb_avail":188436232,"statfs":{"total":193273528320,"available":192958701568,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":313873023},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-2740,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4980,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000973"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":11365,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:05.288100+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:05.288100+0000","last_peered":"2026-02-21T22:24:05.288100+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:05.288100+0000","last_undegraded":"2026-02-21T22:24:05.288100+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11724,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1358},{"shards":"2,3","objects":5862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":24509,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986189,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117007,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153261,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":156939441},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:12.100 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:12.282 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:12.282 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:12.291 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2026-02-21T22:24:12.233714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307576,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306644,"kb_avail":188436104,"statfs":{"total":193273528320,"available":192958570496,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":314004095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-800,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4010,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001204"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":17125,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:10.288358+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:10.288358+0000","last_peered":"2026-02-21T22:24:10.288358+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:10.288358+0000","last_undegraded":"2026-02-21T22:24:10.288358+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7884,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3278},{"shards":"2,3","objects":3942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986190,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117008,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153852,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153389,"kb_avail":94217988,"statfs":{"total":96636764160,"available":96479219712,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":157070513},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:12.291 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:12.478 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:12.479 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:12.489 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2026-02-21T22:24:12.233714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307576,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306644,"kb_avail":188436104,"statfs":{"total":193273528320,"available":192958570496,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":314004095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-800,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4010,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001204"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":17125,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:10.288358+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:10.288358+0000","last_peered":"2026-02-21T22:24:10.288358+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:10.288358+0000","last_undegraded":"2026-02-21T22:24:10.288358+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7884,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3278},{"shards":"2,3","objects":3942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986190,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117008,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153852,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153389,"kb_avail":94217988,"statfs":{"total":96636764160,"available":96479219712,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":157070513},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:12.491 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:12.491 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7382, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426516+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:24:02.426516+0000', 'last_peered': '2026-02-21T22:24:02.426516+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:02.426516+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:12.491 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:12.491 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7211, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426252+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:02.426252+0000', 'last_peered': '2026-02-21T22:24:02.426252+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:02.426252+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:12.492 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:24:12.492 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 17125, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:10.288358+0000', 'last_change': '2026-02-21T22:24:01.205860+0000', 'last_active': '2026-02-21T22:24:10.288358+0000', 'last_peered': '2026-02-21T22:24:10.288358+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:24:10.288358+0000', 'last_undegraded': '2026-02-21T22:24:10.288358+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 7884, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3279, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 3278}, {'shards': '2,3', 'objects': 3942}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:12.492 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:12.711 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:12.711 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12785/115430 objects degraded (11.076%), 2 pgs degraded, 3 pgs undersized","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1157,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":41,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":314957824,"bytes_avail":192958570496,"bytes_total":193273528320,"degraded_objects":12785,"degraded_total":115430,"degraded_ratio":0.11075976782465563,"misplaced_objects":20669,"misplaced_total":115430,"misplaced_ratio":0.17906090271160011,"recovering_objects_per_sec":400,"recovering_bytes_per_sec":1520,"recovering_keys_per_sec":0,"num_objects_recovered":4010,"num_bytes_recovered":15206,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:12.722 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:12.722 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:12.911 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:12.911 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:12.923 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2026-02-21T22:24:12.233714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307576,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306644,"kb_avail":188436104,"statfs":{"total":193273528320,"available":192958570496,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":314004095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-800,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4010,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001204"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":17125,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:10.288358+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:10.288358+0000","last_peered":"2026-02-21T22:24:10.288358+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:10.288358+0000","last_undegraded":"2026-02-21T22:24:10.288358+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7884,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3278},{"shards":"2,3","objects":3942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986190,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117008,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153852,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153389,"kb_avail":94217988,"statfs":{"total":96636764160,"available":96479219712,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":157070513},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:15.925 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:16.113 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:16.113 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:16.124 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2026-02-21T22:24:14.233972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":307576,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":306644,"kb_avail":188436104,"statfs":{"total":193273528320,"available":192958570496,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":314004095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-800,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4010,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.808569"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":17125,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:10.288358+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:10.288358+0000","last_peered":"2026-02-21T22:24:10.288358+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:10.288358+0000","last_undegraded":"2026-02-21T22:24:10.288358+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7884,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3278},{"shards":"2,3","objects":3942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":20669,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39338,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986190,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117008,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153724,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":153255,"kb_avail":94218116,"statfs":{"total":96636764160,"available":96479350784,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":156933582},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153852,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":153389,"kb_avail":94217988,"statfs":{"total":96636764160,"available":96479219712,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":157070513},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:16.125 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:16.313 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:16.314 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:16.324 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":617,"stamp":"2026-02-21T22:24:16.234221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379640,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":378708,"kb_avail":188364040,"statfs":{"total":193273528320,"available":192884776960,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":387797631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-6060,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6640,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":25015,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:15.288959+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:15.288959+0000","last_peered":"2026-02-21T22:24:15.288959+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:15.288959+0000","last_undegraded":"2026-02-21T22:24:15.288959+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2624,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5908},{"shards":"2,3","objects":1312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986191,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117009,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":189351,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":193895886},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:16.325 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:16.325 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7382, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426516+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:24:02.426516+0000', 'last_peered': '2026-02-21T22:24:02.426516+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:02.426516+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:16.326 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:16.326 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7211, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426252+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:02.426252+0000', 'last_peered': '2026-02-21T22:24:02.426252+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:02.426252+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:16.326 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:24:16.326 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 25015, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:15.288959+0000', 'last_change': '2026-02-21T22:24:01.205860+0000', 'last_active': '2026-02-21T22:24:15.288959+0000', 'last_peered': '2026-02-21T22:24:15.288959+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:24:15.288959+0000', 'last_undegraded': '2026-02-21T22:24:15.288959+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2624, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5909, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 5908}, {'shards': '2,3', 'objects': 1312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:16.326 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:16.569 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:16.569 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12785/115430 objects degraded (11.076%), 2 pgs degraded, 3 pgs undersized","count":5},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1161,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":41,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388751360,"bytes_avail":192884776960,"bytes_total":193273528320,"degraded_objects":12785,"degraded_total":115430,"degraded_ratio":0.11075976782465563,"misplaced_objects":15409,"misplaced_total":115430,"misplaced_ratio":0.13349215975049813,"recovering_objects_per_sec":553,"recovering_bytes_per_sec":1267,"recovering_keys_per_sec":0,"num_objects_recovered":6640,"num_bytes_recovered":15206,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:16.581 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:16.581 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:16.762 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:16.763 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:16.772 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":617,"stamp":"2026-02-21T22:24:16.234221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379640,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":378708,"kb_avail":188364040,"statfs":{"total":193273528320,"available":192884776960,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":387797631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7220,"num_objects_misplaced":-6060,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6640,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":25015,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:15.288959+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:15.288959+0000","last_peered":"2026-02-21T22:24:15.288959+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:15.288959+0000","last_undegraded":"2026-02-21T22:24:15.288959+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2624,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5908},{"shards":"2,3","objects":1312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986191,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117009,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":189351,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":193895886},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:19.775 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:19.955 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:19.955 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:19.964 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2026-02-21T22:24:18.295479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379640,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":378708,"kb_avail":188364040,"statfs":{"total":193273528320,"available":192884776960,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":387797631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-9100,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.062018"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":25015,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:15.288959+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:15.288959+0000","last_peered":"2026-02-21T22:24:15.288959+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:15.288959+0000","last_undegraded":"2026-02-21T22:24:15.288959+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2624,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5908},{"shards":"2,3","objects":1312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986191,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117009,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":189351,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":193895886},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:19.966 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:20.150 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:20.150 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:20.160 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2026-02-21T22:24:18.295479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379640,"kb_used_data":856,"kb_used_omap":43,"kb_used_meta":378708,"kb_avail":188364040,"statfs":{"total":193273528320,"available":192884776960,"internally_reserved":0,"allocated":876544,"data_stored":588744,"data_compressed":84414,"data_compressed_allocated":90112,"data_compressed_original":270336,"omap_allocated":44417,"internal_metadata":387797631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-9100,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.062018"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7310,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426335+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:02.426335+0000","last_peered":"2026-02-21T22:24:02.426335+0000","last_clean":"2026-02-21T22:24:02.426335+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:02.426335+0000","last_undegraded":"2026-02-21T22:24:02.426335+0000","last_fullsized":"2026-02-21T22:24:02.426335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":25015,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:15.288959+0000","last_change":"2026-02-21T22:24:01.205860+0000","last_active":"2026-02-21T22:24:15.288959+0000","last_peered":"2026-02-21T22:24:15.288959+0000","last_clean":"2026-02-21T22:22:20.929230+0000","last_became_active":"2026-02-21T22:22:58.399799+0000","last_became_peered":"2026-02-21T22:22:58.399799+0000","last_unstale":"2026-02-21T22:24:15.288959+0000","last_undegraded":"2026-02-21T22:24:15.288959+0000","last_fullsized":"2026-02-21T22:22:58.343404+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2624,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5908},{"shards":"2,3","objects":1312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7206,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.426281+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:02.426281+0000","last_peered":"2026-02-21T22:24:02.426281+0000","last_clean":"2026-02-21T22:24:02.426281+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:02.426281+0000","last_undegraded":"2026-02-21T22:24:02.426281+0000","last_fullsized":"2026-02-21T22:24:02.426281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":15409,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41968,"num_bytes_recovered":167304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":912484,"data_compressed":230220,"data_compressed_allocated":245760,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":13,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986191,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117009,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":24,"kb_used_meta":189351,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":25138,"internal_metadata":193895886},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:20.161 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:20.161 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 7382, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426516+0000', 'last_change': '2026-02-21T22:23:08.492595+0000', 'last_active': '2026-02-21T22:24:02.426516+0000', 'last_peered': '2026-02-21T22:24:02.426516+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:02.426516+0000', 'last_undegraded': '2026-02-21T22:23:08.448684+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 5654, 'num_objects_misplaced': 5654, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 5654}, {'shards': '3', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:20.162 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:20.162 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7211, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:02.426252+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:02.426252+0000', 'last_peered': '2026-02-21T22:24:02.426252+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:02.426252+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:20.162 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-21T22:24:20.162 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'7220", 'reported_seq': 25015, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:15.288959+0000', 'last_change': '2026-02-21T22:24:01.205860+0000', 'last_active': '2026-02-21T22:24:15.288959+0000', 'last_peered': '2026-02-21T22:24:15.288959+0000', 'last_clean': '2026-02-21T22:22:20.929230+0000', 'last_became_active': '2026-02-21T22:22:58.399799+0000', 'last_became_peered': '2026-02-21T22:22:58.399799+0000', 'last_unstale': '2026-02-21T22:24:15.288959+0000', 'last_undegraded': '2026-02-21T22:24:15.288959+0000', 'last_fullsized': '2026-02-21T22:22:58.343404+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7220, 'log_dups_size': 0, 'ondisk_log_size': 7220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2624, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7219, 'num_write_kb': 7247, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5909, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 5908}, {'shards': '2,3', 'objects': 1312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:20.162 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:20.386 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:20.386 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12785/115430 objects degraded (11.076%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1165,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":43,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388784128,"bytes_avail":192884744192,"bytes_total":193273528320,"degraded_objects":12785,"degraded_total":115430,"degraded_ratio":0.11075976782465563,"misplaced_objects":12785,"misplaced_total":115430,"misplaced_ratio":0.11075976782465563,"recovering_objects_per_sec":492,"recovering_bytes_per_sec":3801,"recovering_keys_per_sec":0,"num_objects_recovered":3942,"num_bytes_recovered":30412,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:20.397 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:20.397 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:20.572 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:20.573 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:20.582 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2026-02-21T22:24:20.234704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":12785,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43280,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379672,"kb_used_data":888,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188364008,"statfs":{"total":193273528320,"available":192884744192,"internally_reserved":0,"allocated":909312,"data_stored":625657,"data_compressed":92088,"data_compressed_allocated":98304,"data_compressed_original":294912,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-7884,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000990"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7382,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426516+0000","last_change":"2026-02-21T22:23:08.492595+0000","last_active":"2026-02-21T22:24:02.426516+0000","last_peered":"2026-02-21T22:24:02.426516+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:02.426516+0000","last_undegraded":"2026-02-21T22:23:08.448684+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5654,"num_objects_misplaced":5654,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":5654},{"shards":"3","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7211,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:02.426252+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:02.426252+0000","last_peered":"2026-02-21T22:24:02.426252+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:02.426252+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7335,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427025+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:02.427025+0000","last_peered":"2026-02-21T22:24:02.427025+0000","last_clean":"2026-02-21T22:24:02.427025+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:02.427025+0000","last_undegraded":"2026-02-21T22:24:02.427025+0000","last_fullsized":"2026-02-21T22:24:02.427025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7274,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.427104+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:02.427104+0000","last_peered":"2026-02-21T22:24:02.427104+0000","last_clean":"2026-02-21T22:24:02.427104+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:02.427104+0000","last_undegraded":"2026-02-21T22:24:02.427104+0000","last_fullsized":"2026-02-21T22:24:02.427104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7240,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-21T22:24:02.430402+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:02.430402+0000","last_peered":"2026-02-21T22:24:02.430402+0000","last_clean":"2026-02-21T22:24:02.430402+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:02.430402+0000","last_undegraded":"2026-02-21T22:24:02.430402+0000","last_fullsized":"2026-02-21T22:24:02.430402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12785,"num_objects_misplaced":12785,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43280,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":942896,"data_compressed":237894,"data_compressed_allocated":253952,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986191,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117009,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189820,"kb_used_data":428,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94182020,"statfs":{"total":96636764160,"available":96442388480,"internally_reserved":0,"allocated":438272,"data_stored":294372,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":167285,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:23.585 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:23.766 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:23.766 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:23.776 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":622,"stamp":"2026-02-21T22:24:22.234991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":15733,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44633,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379704,"kb_used_data":920,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188363976,"statfs":{"total":193273528320,"available":192884711424,"internally_reserved":0,"allocated":942080,"data_stored":662570,"data_compressed":99762,"data_compressed_allocated":106496,"data_compressed_original":319488,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5654,"num_objects_misplaced":-4936,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5295,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001019"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":11446,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:20.356991+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:20.356991+0000","last_peered":"2026-02-21T22:24:20.356991+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:20.356991+0000","last_undegraded":"2026-02-21T22:24:20.356991+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8602,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":4301},{"shards":"3","objects":3005}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":15733,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44633,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":973308,"data_compressed":245568,"data_compressed_allocated":262144,"data_compressed_original":786432,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986192,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117010,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:23.777 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:23.968 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:23.969 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:23.979 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":622,"stamp":"2026-02-21T22:24:22.234991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":15733,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44633,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379704,"kb_used_data":920,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188363976,"statfs":{"total":193273528320,"available":192884711424,"internally_reserved":0,"allocated":942080,"data_stored":662570,"data_compressed":99762,"data_compressed_allocated":106496,"data_compressed_original":319488,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5654,"num_objects_misplaced":-4936,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5295,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001019"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":11446,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:20.356991+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:20.356991+0000","last_peered":"2026-02-21T22:24:20.356991+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:20.356991+0000","last_undegraded":"2026-02-21T22:24:20.356991+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8602,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":4301},{"shards":"3","objects":3005}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":15733,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44633,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":973308,"data_compressed":245568,"data_compressed_allocated":262144,"data_compressed_original":786432,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986192,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117010,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:23.981 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:23.981 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 11446, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:20.356991+0000', 'last_change': '2026-02-21T22:24:16.958750+0000', 'last_active': '2026-02-21T22:24:20.356991+0000', 'last_peered': '2026-02-21T22:24:20.356991+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:20.356991+0000', 'last_undegraded': '2026-02-21T22:24:20.356991+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 8602, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1353, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 4301}, {'shards': '3', 'objects': 3005}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:23.982 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:23.982 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7215, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:18.296249+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:18.296249+0000', 'last_peered': '2026-02-21T22:24:18.296249+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:18.296249+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:23.982 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:24.208 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:24.209 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7131/115430 objects degraded (6.178%), 1 pg degraded, 2 pgs undersized","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1169,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":43,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388816896,"bytes_avail":192884711424,"bytes_total":193273528320,"degraded_objects":7131,"degraded_total":115430,"degraded_ratio":0.061777700771030064,"misplaced_objects":15733,"misplaced_total":115430,"misplaced_ratio":0.13629905570475612,"recovering_objects_per_sec":661,"recovering_bytes_per_sec":3801,"recovering_keys_per_sec":0,"num_objects_recovered":5295,"num_bytes_recovered":30412,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:24.219 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:24.219 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:24.402 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:24.402 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:24.413 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2026-02-21T22:24:24.235232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":15733,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44633,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379704,"kb_used_data":920,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188363976,"statfs":{"total":193273528320,"available":192884711424,"internally_reserved":0,"allocated":942080,"data_stored":662570,"data_compressed":99762,"data_compressed_allocated":106496,"data_compressed_original":319488,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5654,"num_objects_misplaced":324,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2665,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001010"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":11446,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:20.356991+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:20.356991+0000","last_peered":"2026-02-21T22:24:20.356991+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:20.356991+0000","last_undegraded":"2026-02-21T22:24:20.356991+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8602,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":4301},{"shards":"3","objects":3005}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":15733,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44633,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":973308,"data_compressed":245568,"data_compressed_allocated":262144,"data_compressed_original":786432,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986192,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117010,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:27.414 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:27.595 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:27.595 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:27.605 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2026-02-21T22:24:26.235466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379704,"kb_used_data":920,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188363976,"statfs":{"total":193273528320,"available":192884711424,"internally_reserved":0,"allocated":942080,"data_stored":662570,"data_compressed":99762,"data_compressed_allocated":106496,"data_compressed_original":319488,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5654,"num_objects_misplaced":-4742,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5198,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.942528"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":19045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:25.357502+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:25.357502+0000","last_peered":"2026-02-21T22:24:25.357502+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:25.357502+0000","last_undegraded":"2026-02-21T22:24:25.357502+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3536,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":1768},{"shards":"3","objects":5538}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":973308,"data_compressed":245568,"data_compressed_allocated":262144,"data_compressed_original":786432,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986193,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117011,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:27.607 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:27.789 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:27.789 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:27.799 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2026-02-21T22:24:26.235466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379704,"kb_used_data":920,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188363976,"statfs":{"total":193273528320,"available":192884711424,"internally_reserved":0,"allocated":942080,"data_stored":662570,"data_compressed":99762,"data_compressed_allocated":106496,"data_compressed_original":319488,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5654,"num_objects_misplaced":-4742,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5198,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.942528"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":19045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:25.357502+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:25.357502+0000","last_peered":"2026-02-21T22:24:25.357502+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:25.357502+0000","last_undegraded":"2026-02-21T22:24:25.357502+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3536,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":1768},{"shards":"3","objects":5538}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":973308,"data_compressed":245568,"data_compressed_allocated":262144,"data_compressed_original":786432,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986193,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117011,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:27.799 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:27.799 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 19045, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:25.357502+0000', 'last_change': '2026-02-21T22:24:16.958750+0000', 'last_active': '2026-02-21T22:24:25.357502+0000', 'last_peered': '2026-02-21T22:24:25.357502+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:25.357502+0000', 'last_undegraded': '2026-02-21T22:24:25.357502+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3536, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3886, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 1768}, {'shards': '3', 'objects': 5538}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:27.800 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:27.800 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7215, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:18.296249+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:18.296249+0000', 'last_peered': '2026-02-21T22:24:18.296249+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:18.296249+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:27.800 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:28.023 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:28.023 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7131/115430 objects degraded (6.178%), 1 pg degraded, 2 pgs undersized","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1172,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":43,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388816896,"bytes_avail":192884711424,"bytes_total":193273528320,"degraded_objects":7131,"degraded_total":115430,"degraded_ratio":0.061777700771030064,"misplaced_objects":10667,"misplaced_total":115430,"misplaced_ratio":0.09241098501256173,"recovering_objects_per_sec":581,"recovering_bytes_per_sec":3400,"recovering_keys_per_sec":0,"num_objects_recovered":5198,"num_bytes_recovered":30412,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:28.035 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:28.035 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:28.218 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:28.218 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:28.228 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2026-02-21T22:24:26.235466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379704,"kb_used_data":920,"kb_used_omap":44,"kb_used_meta":378707,"kb_avail":188363976,"statfs":{"total":193273528320,"available":192884711424,"internally_reserved":0,"allocated":942080,"data_stored":662570,"data_compressed":99762,"data_compressed_allocated":106496,"data_compressed_original":319488,"omap_allocated":45719,"internal_metadata":387796329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5654,"num_objects_misplaced":-4742,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5198,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.942528"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7314,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296149+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:18.296149+0000","last_peered":"2026-02-21T22:24:18.296149+0000","last_clean":"2026-02-21T22:24:18.296149+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:18.296149+0000","last_undegraded":"2026-02-21T22:24:18.296149+0000","last_fullsized":"2026-02-21T22:24:18.296149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":19045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:25.357502+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:25.357502+0000","last_peered":"2026-02-21T22:24:25.357502+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:25.357502+0000","last_undegraded":"2026-02-21T22:24:25.357502+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3536,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":1768},{"shards":"3","objects":5538}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7267,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.300852+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:18.300852+0000","last_peered":"2026-02-21T22:24:18.300852+0000","last_clean":"2026-02-21T22:24:18.300852+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:18.300852+0000","last_undegraded":"2026-02-21T22:24:18.300852+0000","last_fullsized":"2026-02-21T22:24:18.300852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7210,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.296135+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:18.296135+0000","last_peered":"2026-02-21T22:24:18.296135+0000","last_clean":"2026-02-21T22:24:18.296135+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:18.296135+0000","last_undegraded":"2026-02-21T22:24:18.296135+0000","last_fullsized":"2026-02-21T22:24:18.296135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":973308,"data_compressed":245568,"data_compressed_allocated":262144,"data_compressed_original":786432,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986193,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117011,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":25,"kb_used_meta":189350,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":26440,"internal_metadata":193894584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:31.231 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:31.422 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:31.422 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:31.433 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2026-02-21T22:24:30.235954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379752,"kb_used_data":968,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363928,"statfs":{"total":193273528320,"available":192884662272,"internally_reserved":0,"allocated":991232,"data_stored":729845,"data_compressed":115110,"data_compressed_allocated":122880,"data_compressed_original":368640,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5066,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000962"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":19045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:25.357502+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:25.357502+0000","last_peered":"2026-02-21T22:24:25.357502+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:25.357502+0000","last_undegraded":"2026-02-21T22:24:25.357502+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3536,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":1768},{"shards":"3","objects":5538}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":1034132,"data_compressed":260916,"data_compressed_allocated":278528,"data_compressed_original":835584,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986193,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117011,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:31.434 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:31.623 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:31.623 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:31.634 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2026-02-21T22:24:30.235954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379752,"kb_used_data":968,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363928,"statfs":{"total":193273528320,"available":192884662272,"internally_reserved":0,"allocated":991232,"data_stored":729845,"data_compressed":115110,"data_compressed_allocated":122880,"data_compressed_original":368640,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5066,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000962"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":19045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:25.357502+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:25.357502+0000","last_peered":"2026-02-21T22:24:25.357502+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:25.357502+0000","last_undegraded":"2026-02-21T22:24:25.357502+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3536,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":1768},{"shards":"3","objects":5538}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":1034132,"data_compressed":260916,"data_compressed_allocated":278528,"data_compressed_original":835584,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986193,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117011,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:31.635 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-21T22:24:31.635 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'7306", 'reported_seq': 19045, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:25.357502+0000', 'last_change': '2026-02-21T22:24:16.958750+0000', 'last_active': '2026-02-21T22:24:25.357502+0000', 'last_peered': '2026-02-21T22:24:25.357502+0000', 'last_clean': '2026-02-21T22:22:21.168696+0000', 'last_became_active': '2026-02-21T22:23:08.476040+0000', 'last_became_peered': '2026-02-21T22:23:08.476040+0000', 'last_unstale': '2026-02-21T22:24:25.357502+0000', 'last_undegraded': '2026-02-21T22:24:25.357502+0000', 'last_fullsized': '2026-02-21T22:23:08.411676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7306, 'log_dups_size': 0, 'ondisk_log_size': 7306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000221093, 'stat_sum': {'num_bytes': 68126, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3536, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 7362, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3886, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 1768}, {'shards': '3', 'objects': 5538}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T22:24:31.636 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:31.636 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 7215, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2026-02-21T22:24:18.296249+0000', 'last_change': '2026-02-21T22:22:58.403689+0000', 'last_active': '2026-02-21T22:24:18.296249+0000', 'last_peered': '2026-02-21T22:24:18.296249+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:18.296249+0000', 'last_undegraded': '2026-02-21T22:22:58.391763+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7131, 'num_objects_misplaced': 7131, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 7131}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:31.636 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:31.858 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:31.858 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7131/115430 objects degraded (6.178%), 1 pg degraded, 2 pgs undersized","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1176,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388866048,"bytes_avail":192884662272,"bytes_total":193273528320,"degraded_objects":7131,"degraded_total":115430,"degraded_ratio":0.061777700771030064,"misplaced_objects":10667,"misplaced_total":115430,"misplaced_ratio":0.09241098501256173,"recovering_objects_per_sec":316,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:31.868 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:31.869 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:32.049 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:32.049 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:32.059 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2026-02-21T22:24:30.235954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379752,"kb_used_data":968,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363928,"statfs":{"total":193273528320,"available":192884662272,"internally_reserved":0,"allocated":991232,"data_stored":729845,"data_compressed":115110,"data_compressed_allocated":122880,"data_compressed_original":368640,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5066,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000962"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":19045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:25.357502+0000","last_change":"2026-02-21T22:24:16.958750+0000","last_active":"2026-02-21T22:24:25.357502+0000","last_peered":"2026-02-21T22:24:25.357502+0000","last_clean":"2026-02-21T22:22:21.168696+0000","last_became_active":"2026-02-21T22:23:08.476040+0000","last_became_peered":"2026-02-21T22:23:08.476040+0000","last_unstale":"2026-02-21T22:24:25.357502+0000","last_undegraded":"2026-02-21T22:24:25.357502+0000","last_fullsized":"2026-02-21T22:23:08.411676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3536,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":1768},{"shards":"3","objects":5538}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2026-02-21T22:24:18.296249+0000","last_change":"2026-02-21T22:22:58.403689+0000","last_active":"2026-02-21T22:24:18.296249+0000","last_peered":"2026-02-21T22:24:18.296249+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:18.296249+0000","last_undegraded":"2026-02-21T22:22:58.391763+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":7131,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7131}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7339,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297047+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:18.297047+0000","last_peered":"2026-02-21T22:24:18.297047+0000","last_clean":"2026-02-21T22:24:18.297047+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:18.297047+0000","last_undegraded":"2026-02-21T22:24:18.297047+0000","last_fullsized":"2026-02-21T22:24:18.297047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7278,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297214+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:18.297214+0000","last_peered":"2026-02-21T22:24:18.297214+0000","last_clean":"2026-02-21T22:24:18.297214+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:18.297214+0000","last_undegraded":"2026-02-21T22:24:18.297214+0000","last_fullsized":"2026-02-21T22:24:18.297214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7244,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-21T22:24:18.297180+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:18.297180+0000","last_peered":"2026-02-21T22:24:18.297180+0000","last_clean":"2026-02-21T22:24:18.297180+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:18.297180+0000","last_undegraded":"2026-02-21T22:24:18.297180+0000","last_fullsized":"2026-02-21T22:24:18.297180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7131,"num_objects_misplaced":10667,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47166,"num_bytes_recovered":197716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":1034132,"data_compressed":260916,"data_compressed_allocated":278528,"data_compressed_original":835584,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":14,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986193,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117011,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189852,"kb_used_data":460,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181988,"statfs":{"total":96636764160,"available":96442355712,"internally_reserved":0,"allocated":471040,"data_stored":331285,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":197697,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:35.059 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:35.247 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:35.248 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:35.258 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":630,"stamp":"2026-02-21T22:24:34.236449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49811,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379800,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363880,"statfs":{"total":193273528320,"available":192884613120,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":1841,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2645,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000983"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":9851,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:30.289689+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:30.289689+0000","last_peered":"2026-02-21T22:24:30.289689+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:30.289689+0000","last_undegraded":"2026-02-21T22:24:30.289689+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":877},{"shards":"2,3","objects":6254}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49811,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":1094956,"data_compressed":276264,"data_compressed_allocated":294912,"data_compressed_original":884736,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986194,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117012,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:35.260 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:35.448 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:35.449 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:35.459 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":630,"stamp":"2026-02-21T22:24:34.236449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49811,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379800,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363880,"statfs":{"total":193273528320,"available":192884613120,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":1841,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2645,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000983"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":9851,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:30.289689+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:30.289689+0000","last_peered":"2026-02-21T22:24:30.289689+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:30.289689+0000","last_undegraded":"2026-02-21T22:24:30.289689+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":877},{"shards":"2,3","objects":6254}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49811,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":1094956,"data_compressed":276264,"data_compressed_allocated":294912,"data_compressed_original":884736,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986194,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117012,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:35.459 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:35.459 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 9851, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:30.289689+0000', 'last_change': '2026-02-21T22:24:27.664156+0000', 'last_active': '2026-02-21T22:24:30.289689+0000', 'last_peered': '2026-02-21T22:24:30.289689+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:30.289689+0000', 'last_undegraded': '2026-02-21T22:24:30.289689+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 12508, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 877, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 877}, {'shards': '2,3', 'objects': 6254}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:35.459 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:35.680 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:35.680 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg undersized","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1180,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388915200,"bytes_avail":192884613120,"bytes_total":193273528320,"misplaced_objects":12508,"misplaced_total":115430,"misplaced_ratio":0.10836004504894742,"recovering_objects_per_sec":330,"recovering_bytes_per_sec":7602,"recovering_keys_per_sec":0,"num_objects_recovered":2645,"num_bytes_recovered":60824,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:35.691 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:35.691 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:35.871 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:35.872 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:35.881 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":630,"stamp":"2026-02-21T22:24:34.236449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49811,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379800,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363880,"statfs":{"total":193273528320,"available":192884613120,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":1841,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2645,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000983"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":9851,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:30.289689+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:30.289689+0000","last_peered":"2026-02-21T22:24:30.289689+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:30.289689+0000","last_undegraded":"2026-02-21T22:24:30.289689+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":877},{"shards":"2,3","objects":6254}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12508,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49811,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":1094956,"data_compressed":276264,"data_compressed_allocated":294912,"data_compressed_original":884736,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986194,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117012,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:38.884 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:39.065 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:39.066 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:39.077 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2026-02-21T22:24:38.236921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51675,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379800,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363880,"statfs":{"total":193273528320,"available":192884613120,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":-1887,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4509,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.875560"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":15443,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:35.291715+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:35.291715+0000","last_peered":"2026-02-21T22:24:35.291715+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:35.291715+0000","last_undegraded":"2026-02-21T22:24:35.291715+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":2741},{"shards":"2,3","objects":4390}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51675,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":1079750,"data_compressed":272427,"data_compressed_allocated":290816,"data_compressed_original":872448,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986195,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117013,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:39.079 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:39.264 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:39.264 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:39.274 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2026-02-21T22:24:38.236921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51675,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379800,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363880,"statfs":{"total":193273528320,"available":192884613120,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":-1887,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4509,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.875560"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":15443,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:35.291715+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:35.291715+0000","last_peered":"2026-02-21T22:24:35.291715+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:35.291715+0000","last_undegraded":"2026-02-21T22:24:35.291715+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":2741},{"shards":"2,3","objects":4390}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51675,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":1079750,"data_compressed":272427,"data_compressed_allocated":290816,"data_compressed_original":872448,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986195,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117013,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:39.275 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:39.275 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 15443, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:35.291715+0000', 'last_change': '2026-02-21T22:24:27.664156+0000', 'last_active': '2026-02-21T22:24:35.291715+0000', 'last_peered': '2026-02-21T22:24:35.291715+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:35.291715+0000', 'last_undegraded': '2026-02-21T22:24:35.291715+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 8780, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2741, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 2741}, {'shards': '2,3', 'objects': 4390}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:39.275 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:39.496 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:39.496 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg undersized","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1184,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":388915200,"bytes_avail":192884613120,"bytes_total":193273528320,"misplaced_objects":8780,"misplaced_total":115430,"misplaced_ratio":0.076063415056744341,"recovering_objects_per_sec":456,"recovering_bytes_per_sec":6159,"recovering_keys_per_sec":0,"num_objects_recovered":4509,"num_bytes_recovered":60824,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:39.507 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:39.507 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:39.693 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:39.694 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:39.703 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2026-02-21T22:24:38.236921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51675,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":379800,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":378706,"kb_avail":188363880,"statfs":{"total":193273528320,"available":192884613120,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":387795027},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":-1887,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4509,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.875560"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":15443,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:35.291715+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:35.291715+0000","last_peered":"2026-02-21T22:24:35.291715+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:35.291715+0000","last_undegraded":"2026-02-21T22:24:35.291715+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":2741},{"shards":"2,3","objects":4390}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8780,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51675,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":1079750,"data_compressed":272427,"data_compressed_allocated":290816,"data_compressed_original":872448,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986195,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117013,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":189348,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":193893282},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189900,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":189357,"kb_avail":94181940,"statfs":{"total":96636764160,"available":96442306560,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":193901745},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:42.704 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:42.891 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:42.891 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:42.901 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2026-02-21T22:24:42.237382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54093,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":415896,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":414802,"kb_avail":188327784,"statfs":{"total":193273528320,"available":192847650816,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":424757331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":-6723,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6927,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001428"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":22697,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:40.292223+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:40.292223+0000","last_peered":"2026-02-21T22:24:40.292223+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:40.292223+0000","last_undegraded":"2026-02-21T22:24:40.292223+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5159,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5159},{"shards":"2,3","objects":1972}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54093,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":1049338,"data_compressed":264753,"data_compressed_allocated":282624,"data_compressed_original":847872,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986196,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117014,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207948,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":207396,"kb_avail":94163892,"statfs":{"total":96636764160,"available":96423825408,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":212374434},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207948,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":207405,"kb_avail":94163892,"statfs":{"total":96636764160,"available":96423825408,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":212382897},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:42.903 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:43.081 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:43.082 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:43.090 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2026-02-21T22:24:42.237382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54093,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":415896,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":414802,"kb_avail":188327784,"statfs":{"total":193273528320,"available":192847650816,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":424757331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7131,"num_objects_misplaced":-6723,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6927,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001428"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":22697,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:40.292223+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:40.292223+0000","last_peered":"2026-02-21T22:24:40.292223+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:40.292223+0000","last_undegraded":"2026-02-21T22:24:40.292223+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5159,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5159},{"shards":"2,3","objects":1972}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54093,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":1049338,"data_compressed":264753,"data_compressed_allocated":282624,"data_compressed_original":847872,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986196,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117014,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207948,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":207396,"kb_avail":94163892,"statfs":{"total":96636764160,"available":96423825408,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":212374434},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207948,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":207405,"kb_avail":94163892,"statfs":{"total":96636764160,"available":96423825408,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":212382897},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:43.090 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-21T22:24:43.090 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'7131", 'reported_seq': 22697, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-21T22:24:40.292223+0000', 'last_change': '2026-02-21T22:24:27.664156+0000', 'last_active': '2026-02-21T22:24:40.292223+0000', 'last_peered': '2026-02-21T22:24:40.292223+0000', 'last_clean': '2026-02-21T22:22:22.353928+0000', 'last_became_active': '2026-02-21T22:22:58.401030+0000', 'last_became_peered': '2026-02-21T22:22:58.401030+0000', 'last_unstale': '2026-02-21T22:24:40.292223+0000', 'last_undegraded': '2026-02-21T22:24:40.292223+0000', 'last_fullsized': '2026-02-21T22:22:58.343486+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'last_clean_scrub_stamp': '2026-02-21T22:05:04.195147+0000', 'objects_scrubbed': 0, 'log_size': 7131, 'log_dups_size': 0, 'ondisk_log_size': 7131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000211775, 'stat_sum': {'num_bytes': 37564, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3944, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 7159, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5159, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 5159}, {'shards': '2,3', 'objects': 1972}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-21T22:24:43.091 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T22:24:43.306 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:43.306 INFO:teuthology.orchestra.run.trial060.stdout:{"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg undersized","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1188,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1771712586,"num_in_osds":2,"osd_in_since":1771712543,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":57715,"data_bytes":346650,"bytes_used":425877504,"bytes_avail":192847650816,"bytes_total":193273528320,"misplaced_objects":3944,"misplaced_total":115430,"misplaced_ratio":0.034167893961708391,"recovering_objects_per_sec":577,"recovering_bytes_per_sec":5068,"recovering_keys_per_sec":0,"num_objects_recovered":6927,"num_bytes_recovered":60824,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-21T22:04:55:178453+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-21T22:24:06.233031+0000","services":{}},"progress_events":{}} 2026-02-21T22:24:43.314 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-21T22:24:43.314 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:43.489 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:43.490 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:43.497 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":635,"stamp":"2026-02-21T22:24:43.384141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54093,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":415896,"kb_used_data":1016,"kb_used_omap":45,"kb_used_meta":414802,"kb_avail":188327784,"statfs":{"total":193273528320,"available":192847650816,"internally_reserved":0,"allocated":1040384,"data_stored":797120,"data_compressed":130458,"data_compressed_allocated":139264,"data_compressed_original":417792,"omap_allocated":47021,"internal_metadata":424757331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8564,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.147923"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365126+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:29.365126+0000","last_peered":"2026-02-21T22:24:29.365126+0000","last_clean":"2026-02-21T22:24:29.365126+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:29.365126+0000","last_undegraded":"2026-02-21T22:24:29.365126+0000","last_fullsized":"2026-02-21T22:24:29.365126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7372,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.368823+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:29.368823+0000","last_peered":"2026-02-21T22:24:29.368823+0000","last_clean":"2026-02-21T22:24:29.368823+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:29.368823+0000","last_undegraded":"2026-02-21T22:24:29.368823+0000","last_fullsized":"2026-02-21T22:24:29.368823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":22697,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-21T22:24:40.292223+0000","last_change":"2026-02-21T22:24:27.664156+0000","last_active":"2026-02-21T22:24:40.292223+0000","last_peered":"2026-02-21T22:24:40.292223+0000","last_clean":"2026-02-21T22:22:22.353928+0000","last_became_active":"2026-02-21T22:22:58.401030+0000","last_became_peered":"2026-02-21T22:22:58.401030+0000","last_unstale":"2026-02-21T22:24:40.292223+0000","last_undegraded":"2026-02-21T22:24:40.292223+0000","last_fullsized":"2026-02-21T22:22:58.343486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5159,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5159},{"shards":"2,3","objects":1972}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364862+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:29.364862+0000","last_peered":"2026-02-21T22:24:29.364862+0000","last_clean":"2026-02-21T22:24:29.364862+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:29.364862+0000","last_undegraded":"2026-02-21T22:24:29.364862+0000","last_fullsized":"2026-02-21T22:24:29.364862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7271,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365140+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:29.365140+0000","last_peered":"2026-02-21T22:24:29.365140+0000","last_clean":"2026-02-21T22:24:29.365140+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:29.365140+0000","last_undegraded":"2026-02-21T22:24:29.365140+0000","last_fullsized":"2026-02-21T22:24:29.365140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7282,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365074+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:29.365074+0000","last_peered":"2026-02-21T22:24:29.365074+0000","last_clean":"2026-02-21T22:24:29.365074+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:29.365074+0000","last_undegraded":"2026-02-21T22:24:29.365074+0000","last_fullsized":"2026-02-21T22:24:29.365074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.364939+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:29.364939+0000","last_peered":"2026-02-21T22:24:29.364939+0000","last_clean":"2026-02-21T22:24:29.364939+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:29.364939+0000","last_undegraded":"2026-02-21T22:24:29.364939+0000","last_fullsized":"2026-02-21T22:24:29.364939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7214,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-21T22:24:29.365107+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:29.365107+0000","last_peered":"2026-02-21T22:24:29.365107+0000","last_clean":"2026-02-21T22:24:29.365107+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:29.365107+0000","last_undegraded":"2026-02-21T22:24:29.365107+0000","last_fullsized":"2026-02-21T22:24:29.365107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3944,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54093,"num_bytes_recovered":258540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":1049338,"data_compressed":264753,"data_compressed_allocated":282624,"data_compressed_original":847872,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986196,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117014,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207948,"kb_used_data":508,"kb_used_omap":27,"kb_used_meta":207396,"kb_avail":94163892,"statfs":{"total":96636764160,"available":96423825408,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":27742,"internal_metadata":212374434},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207948,"kb_used_data":508,"kb_used_omap":18,"kb_used_meta":207405,"kb_avail":94163892,"statfs":{"total":96636764160,"available":96423825408,"internally_reserved":0,"allocated":520192,"data_stored":398560,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":19279,"internal_metadata":212382897},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":258521,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:46.500 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:46.675 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:46.676 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:46.684 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":638,"stamp":"2026-02-21T22:24:46.237950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":452064,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":450887,"kb_avail":188291616,"statfs":{"total":193273528320,"available":192810614784,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":461708568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4390,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001028"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7322,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385144+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:44.385144+0000","last_peered":"2026-02-21T22:24:44.385144+0000","last_clean":"2026-02-21T22:24:44.385144+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:44.385144+0000","last_undegraded":"2026-02-21T22:24:44.385144+0000","last_fullsized":"2026-02-21T22:24:44.385144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7376,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385850+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:44.385850+0000","last_peered":"2026-02-21T22:24:44.385850+0000","last_clean":"2026-02-21T22:24:44.385850+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:44.385850+0000","last_undegraded":"2026-02-21T22:24:44.385850+0000","last_fullsized":"2026-02-21T22:24:44.385850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7188,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.388806+0000","last_change":"2026-02-21T22:24:44.388766+0000","last_active":"2026-02-21T22:24:44.388806+0000","last_peered":"2026-02-21T22:24:44.388806+0000","last_clean":"2026-02-21T22:24:44.388806+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:24:44.388806+0000","last_undegraded":"2026-02-21T22:24:44.388806+0000","last_fullsized":"2026-02-21T22:24:44.388806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:57:58.726085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7347,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385802+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:44.385802+0000","last_peered":"2026-02-21T22:24:44.385802+0000","last_clean":"2026-02-21T22:24:44.385802+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:44.385802+0000","last_undegraded":"2026-02-21T22:24:44.385802+0000","last_fullsized":"2026-02-21T22:24:44.385802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7275,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385165+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:44.385165+0000","last_peered":"2026-02-21T22:24:44.385165+0000","last_clean":"2026-02-21T22:24:44.385165+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:44.385165+0000","last_undegraded":"2026-02-21T22:24:44.385165+0000","last_fullsized":"2026-02-21T22:24:44.385165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7286,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385898+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:44.385898+0000","last_peered":"2026-02-21T22:24:44.385898+0000","last_clean":"2026-02-21T22:24:44.385898+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:44.385898+0000","last_undegraded":"2026-02-21T22:24:44.385898+0000","last_fullsized":"2026-02-21T22:24:44.385898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7252,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385839+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:44.385839+0000","last_peered":"2026-02-21T22:24:44.385839+0000","last_clean":"2026-02-21T22:24:44.385839+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:44.385839+0000","last_undegraded":"2026-02-21T22:24:44.385839+0000","last_fullsized":"2026-02-21T22:24:44.385839+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7219,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385148+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:44.385148+0000","last_peered":"2026-02-21T22:24:44.385148+0000","last_clean":"2026-02-21T22:24:44.385148+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:44.385148+0000","last_undegraded":"2026-02-21T22:24:44.385148+0000","last_fullsized":"2026-02-21T22:24:44.385148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":1110210,"data_compressed":280101,"data_compressed_allocated":299008,"data_compressed_original":897024,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986197,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117015,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:46.685 INFO:tasks.osd_recovery.ceph_manager:clean! 2026-02-21T22:24:46.685 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T22:24:46.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T22:24:46.700 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:46.877 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:46.877 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:46.886 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":638,"stamp":"2026-02-21T22:24:46.237950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":452064,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":450887,"kb_avail":188291616,"statfs":{"total":193273528320,"available":192810614784,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":461708568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4390,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001028"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7322,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385144+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:44.385144+0000","last_peered":"2026-02-21T22:24:44.385144+0000","last_clean":"2026-02-21T22:24:44.385144+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:44.385144+0000","last_undegraded":"2026-02-21T22:24:44.385144+0000","last_fullsized":"2026-02-21T22:24:44.385144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7376,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385850+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:44.385850+0000","last_peered":"2026-02-21T22:24:44.385850+0000","last_clean":"2026-02-21T22:24:44.385850+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:44.385850+0000","last_undegraded":"2026-02-21T22:24:44.385850+0000","last_fullsized":"2026-02-21T22:24:44.385850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7188,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.388806+0000","last_change":"2026-02-21T22:24:44.388766+0000","last_active":"2026-02-21T22:24:44.388806+0000","last_peered":"2026-02-21T22:24:44.388806+0000","last_clean":"2026-02-21T22:24:44.388806+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:24:44.388806+0000","last_undegraded":"2026-02-21T22:24:44.388806+0000","last_fullsized":"2026-02-21T22:24:44.388806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:57:58.726085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7347,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385802+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:44.385802+0000","last_peered":"2026-02-21T22:24:44.385802+0000","last_clean":"2026-02-21T22:24:44.385802+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:44.385802+0000","last_undegraded":"2026-02-21T22:24:44.385802+0000","last_fullsized":"2026-02-21T22:24:44.385802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7275,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385165+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:44.385165+0000","last_peered":"2026-02-21T22:24:44.385165+0000","last_clean":"2026-02-21T22:24:44.385165+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:44.385165+0000","last_undegraded":"2026-02-21T22:24:44.385165+0000","last_fullsized":"2026-02-21T22:24:44.385165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7286,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385898+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:44.385898+0000","last_peered":"2026-02-21T22:24:44.385898+0000","last_clean":"2026-02-21T22:24:44.385898+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:44.385898+0000","last_undegraded":"2026-02-21T22:24:44.385898+0000","last_fullsized":"2026-02-21T22:24:44.385898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7252,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385839+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:44.385839+0000","last_peered":"2026-02-21T22:24:44.385839+0000","last_clean":"2026-02-21T22:24:44.385839+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:44.385839+0000","last_undegraded":"2026-02-21T22:24:44.385839+0000","last_fullsized":"2026-02-21T22:24:44.385839+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7219,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385148+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:44.385148+0000","last_peered":"2026-02-21T22:24:44.385148+0000","last_clean":"2026-02-21T22:24:44.385148+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:44.385148+0000","last_undegraded":"2026-02-21T22:24:44.385148+0000","last_fullsized":"2026-02-21T22:24:44.385148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":1110210,"data_compressed":280101,"data_compressed_allocated":299008,"data_compressed_original":897024,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986197,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117015,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:46.888 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:47.066 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:47.066 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:47.074 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":638,"stamp":"2026-02-21T22:24:46.237950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":452064,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":450887,"kb_avail":188291616,"statfs":{"total":193273528320,"available":192810614784,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":461708568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4390,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001028"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7322,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385144+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:44.385144+0000","last_peered":"2026-02-21T22:24:44.385144+0000","last_clean":"2026-02-21T22:24:44.385144+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:44.385144+0000","last_undegraded":"2026-02-21T22:24:44.385144+0000","last_fullsized":"2026-02-21T22:24:44.385144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7376,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385850+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:44.385850+0000","last_peered":"2026-02-21T22:24:44.385850+0000","last_clean":"2026-02-21T22:24:44.385850+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:44.385850+0000","last_undegraded":"2026-02-21T22:24:44.385850+0000","last_fullsized":"2026-02-21T22:24:44.385850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7188,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.388806+0000","last_change":"2026-02-21T22:24:44.388766+0000","last_active":"2026-02-21T22:24:44.388806+0000","last_peered":"2026-02-21T22:24:44.388806+0000","last_clean":"2026-02-21T22:24:44.388806+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:24:44.388806+0000","last_undegraded":"2026-02-21T22:24:44.388806+0000","last_fullsized":"2026-02-21T22:24:44.388806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:57:58.726085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7347,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385802+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:44.385802+0000","last_peered":"2026-02-21T22:24:44.385802+0000","last_clean":"2026-02-21T22:24:44.385802+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:44.385802+0000","last_undegraded":"2026-02-21T22:24:44.385802+0000","last_fullsized":"2026-02-21T22:24:44.385802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7275,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385165+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:44.385165+0000","last_peered":"2026-02-21T22:24:44.385165+0000","last_clean":"2026-02-21T22:24:44.385165+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:44.385165+0000","last_undegraded":"2026-02-21T22:24:44.385165+0000","last_fullsized":"2026-02-21T22:24:44.385165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7286,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385898+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:44.385898+0000","last_peered":"2026-02-21T22:24:44.385898+0000","last_clean":"2026-02-21T22:24:44.385898+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:44.385898+0000","last_undegraded":"2026-02-21T22:24:44.385898+0000","last_fullsized":"2026-02-21T22:24:44.385898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7252,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385839+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:44.385839+0000","last_peered":"2026-02-21T22:24:44.385839+0000","last_clean":"2026-02-21T22:24:44.385839+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:44.385839+0000","last_undegraded":"2026-02-21T22:24:44.385839+0000","last_fullsized":"2026-02-21T22:24:44.385839+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7219,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385148+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:44.385148+0000","last_peered":"2026-02-21T22:24:44.385148+0000","last_clean":"2026-02-21T22:24:44.385148+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:44.385148+0000","last_undegraded":"2026-02-21T22:24:44.385148+0000","last_fullsized":"2026-02-21T22:24:44.385148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":1110210,"data_compressed":280101,"data_compressed_allocated":299008,"data_compressed_original":897024,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986197,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117015,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:47.075 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T22:24:47.075 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:47.247 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:47.248 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:47.256 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":638,"stamp":"2026-02-21T22:24:46.237950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":452064,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":450887,"kb_avail":188291616,"statfs":{"total":193273528320,"available":192810614784,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":461708568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4390,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001028"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7322,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385144+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:44.385144+0000","last_peered":"2026-02-21T22:24:44.385144+0000","last_clean":"2026-02-21T22:24:44.385144+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:44.385144+0000","last_undegraded":"2026-02-21T22:24:44.385144+0000","last_fullsized":"2026-02-21T22:24:44.385144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7376,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385850+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:44.385850+0000","last_peered":"2026-02-21T22:24:44.385850+0000","last_clean":"2026-02-21T22:24:44.385850+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:44.385850+0000","last_undegraded":"2026-02-21T22:24:44.385850+0000","last_fullsized":"2026-02-21T22:24:44.385850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7188,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.388806+0000","last_change":"2026-02-21T22:24:44.388766+0000","last_active":"2026-02-21T22:24:44.388806+0000","last_peered":"2026-02-21T22:24:44.388806+0000","last_clean":"2026-02-21T22:24:44.388806+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:24:44.388806+0000","last_undegraded":"2026-02-21T22:24:44.388806+0000","last_fullsized":"2026-02-21T22:24:44.388806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:57:58.726085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7347,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385802+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:44.385802+0000","last_peered":"2026-02-21T22:24:44.385802+0000","last_clean":"2026-02-21T22:24:44.385802+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:44.385802+0000","last_undegraded":"2026-02-21T22:24:44.385802+0000","last_fullsized":"2026-02-21T22:24:44.385802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7275,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385165+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:44.385165+0000","last_peered":"2026-02-21T22:24:44.385165+0000","last_clean":"2026-02-21T22:24:44.385165+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:44.385165+0000","last_undegraded":"2026-02-21T22:24:44.385165+0000","last_fullsized":"2026-02-21T22:24:44.385165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7286,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385898+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:44.385898+0000","last_peered":"2026-02-21T22:24:44.385898+0000","last_clean":"2026-02-21T22:24:44.385898+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:44.385898+0000","last_undegraded":"2026-02-21T22:24:44.385898+0000","last_fullsized":"2026-02-21T22:24:44.385898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7252,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385839+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:44.385839+0000","last_peered":"2026-02-21T22:24:44.385839+0000","last_clean":"2026-02-21T22:24:44.385839+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:44.385839+0000","last_undegraded":"2026-02-21T22:24:44.385839+0000","last_fullsized":"2026-02-21T22:24:44.385839+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7219,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385148+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:44.385148+0000","last_peered":"2026-02-21T22:24:44.385148+0000","last_clean":"2026-02-21T22:24:44.385148+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:44.385148+0000","last_undegraded":"2026-02-21T22:24:44.385148+0000","last_fullsized":"2026-02-21T22:24:44.385148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":1110210,"data_compressed":280101,"data_compressed_allocated":299008,"data_compressed_original":897024,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986197,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117015,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:47.256 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:24:47.433 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:47.433 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":47,"fsid":"ab2fd705-c792-40f6-9c14-b3ae92aaa530","created":"2026-02-21T22:04:55.178617+0000","modified":"2026-02-21T22:24:44.383323+0000","last_up_change":"2026-02-21T22:23:06.345395+0000","last_in_change":"2026-02-21T22:22:23.349920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T22:05:03.548523+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.5,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"171d695f-2570-47af-aab5-ae0bd86f3717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6805","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6808","nonce":1304182596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6811","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6814","nonce":1304182596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6823","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6825","nonce":1304182596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6817","nonce":1304182596},{"type":"v1","addr":"10.20.193.60:6820","nonce":1304182596}]},"public_addr":"10.20.193.60:6808/1304182596","cluster_addr":"10.20.193.60:6814/1304182596","heartbeat_back_addr":"10.20.193.60:6825/1304182596","heartbeat_front_addr":"10.20.193.60:6820/1304182596","state":["exists","up"]},{"osd":1,"uuid":"638b1ff9-5ba8-44f4-9c59-2c8cceb4c8d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6826","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6827","nonce":2501220781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6828","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6829","nonce":2501220781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6832","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6833","nonce":2501220781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6830","nonce":2501220781},{"type":"v1","addr":"10.20.193.60:6831","nonce":2501220781}]},"public_addr":"10.20.193.60:6827/2501220781","cluster_addr":"10.20.193.60:6829/2501220781","heartbeat_back_addr":"10.20.193.60:6833/2501220781","heartbeat_front_addr":"10.20.193.60:6831/2501220781","state":["exists","up"]},{"osd":2,"uuid":"fd94ed65-fa21-4566-aa25-5e039e1b7d32","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":27,"up_thru":45,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":4232863995},{"type":"v1","addr":"10.20.193.60:6803","nonce":4232863995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":4232863995},{"type":"v1","addr":"10.20.193.60:6806","nonce":4232863995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":4232863995},{"type":"v1","addr":"10.20.193.60:6812","nonce":4232863995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6807","nonce":4232863995},{"type":"v1","addr":"10.20.193.60:6809","nonce":4232863995}]},"public_addr":"10.20.193.60:6803/4232863995","cluster_addr":"10.20.193.60:6806/4232863995","heartbeat_back_addr":"10.20.193.60:6812/4232863995","heartbeat_front_addr":"10.20.193.60:6809/4232863995","state":["exists","up"]},{"osd":3,"uuid":"b5d2feef-3a3b-48bd-a117-8263471172c5","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":43,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6813","nonce":3051683365},{"type":"v1","addr":"10.20.193.60:6815","nonce":3051683365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":3051683365},{"type":"v1","addr":"10.20.193.60:6818","nonce":3051683365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":3051683365},{"type":"v1","addr":"10.20.193.60:6824","nonce":3051683365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6819","nonce":3051683365},{"type":"v1","addr":"10.20.193.60:6821","nonce":3051683365}]},"public_addr":"10.20.193.60:6815/3051683365","cluster_addr":"10.20.193.60:6818/3051683365","heartbeat_back_addr":"10.20.193.60:6824/3051683365","heartbeat_front_addr":"10.20.193.60:6821/3051683365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:04:57.809661+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:04:57.827273+0000","dead_epoch":0},{"osd":2,"down_stamp":"2026-02-21T22:22:46.378619+0000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":65536,"last_purged_snaps_scrub":"2026-02-21T22:04:57.826189+0000","dead_epoch":0},{"osd":3,"down_stamp":"2026-02-21T22:22:46.378619+0000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":65536,"last_purged_snaps_scrub":"2026-02-21T22:04:57.857774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:24:48.444 INFO:tasks.ceph:Scrubbing osd.0 2026-02-21T22:24:48.444 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:24:48.523 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:24:48.524 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:24:48.524 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:24:48.529 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-21T22:24:48.706 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 0 to deep-scrub 2026-02-21T22:24:48.715 INFO:tasks.ceph:Scrubbing osd.1 2026-02-21T22:24:48.716 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:24:48.794 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:24:48.795 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:24:48.795 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:24:48.801 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-21T22:24:48.977 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 1 to deep-scrub 2026-02-21T22:24:48.985 INFO:tasks.ceph:Scrubbing osd.2 2026-02-21T22:24:48.985 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:24:49.071 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:24:49.071 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:24:49.071 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:24:49.077 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-21T22:24:49.264 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 2 to deep-scrub 2026-02-21T22:24:49.275 INFO:tasks.ceph:Scrubbing osd.3 2026-02-21T22:24:49.275 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:24:49.358 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-21T22:24:49.359 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:24:49.359 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-21T22:24:49.367 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-21T22:24:49.559 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 3 to deep-scrub 2026-02-21T22:24:49.570 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:24:49.753 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:24:49.753 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:24:49.765 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2026-02-21T22:24:48.238169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":452064,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":450887,"kb_avail":188291616,"statfs":{"total":193273528320,"available":192810614784,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":461708568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8780,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4390,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001021"},"pg_stats":[{"pgid":"1.7","version":"17'7260","reported_seq":7322,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385144+0000","last_change":"2026-02-21T22:23:15.368259+0000","last_active":"2026-02-21T22:24:44.385144+0000","last_peered":"2026-02-21T22:24:44.385144+0000","last_clean":"2026-02-21T22:24:44.385144+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:24:44.385144+0000","last_undegraded":"2026-02-21T22:24:44.385144+0000","last_fullsized":"2026-02-21T22:24:44.385144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:06.751936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'7306","reported_seq":7376,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385850+0000","last_change":"2026-02-21T22:24:29.368766+0000","last_active":"2026-02-21T22:24:44.385850+0000","last_peered":"2026-02-21T22:24:44.385850+0000","last_clean":"2026-02-21T22:24:44.385850+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:24:44.385850+0000","last_undegraded":"2026-02-21T22:24:44.385850+0000","last_fullsized":"2026-02-21T22:24:44.385850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7306,"log_dups_size":0,"ondisk_log_size":7306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:20:29.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'7131","reported_seq":7188,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.388806+0000","last_change":"2026-02-21T22:24:44.388766+0000","last_active":"2026-02-21T22:24:44.388806+0000","last_peered":"2026-02-21T22:24:44.388806+0000","last_clean":"2026-02-21T22:24:44.388806+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:24:44.388806+0000","last_undegraded":"2026-02-21T22:24:44.388806+0000","last_fullsized":"2026-02-21T22:24:44.388806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7131,"log_dups_size":0,"ondisk_log_size":7131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:57:58.726085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'7260","reported_seq":7347,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385802+0000","last_change":"2026-02-21T22:23:30.861843+0000","last_active":"2026-02-21T22:24:44.385802+0000","last_peered":"2026-02-21T22:24:44.385802+0000","last_clean":"2026-02-21T22:24:44.385802+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:24:44.385802+0000","last_undegraded":"2026-02-21T22:24:44.385802+0000","last_fullsized":"2026-02-21T22:24:44.385802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:57:52.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'7220","reported_seq":7275,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385165+0000","last_change":"2026-02-21T22:24:18.300794+0000","last_active":"2026-02-21T22:24:44.385165+0000","last_peered":"2026-02-21T22:24:44.385165+0000","last_clean":"2026-02-21T22:24:44.385165+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:24:44.385165+0000","last_undegraded":"2026-02-21T22:24:44.385165+0000","last_fullsized":"2026-02-21T22:24:44.385165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:06:56.733650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'7229","reported_seq":7286,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385898+0000","last_change":"2026-02-21T22:23:47.300379+0000","last_active":"2026-02-21T22:24:44.385898+0000","last_peered":"2026-02-21T22:24:44.385898+0000","last_clean":"2026-02-21T22:24:44.385898+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:24:44.385898+0000","last_undegraded":"2026-02-21T22:24:44.385898+0000","last_fullsized":"2026-02-21T22:24:44.385898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7229,"log_dups_size":0,"ondisk_log_size":7229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:41:49.136874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'7173","reported_seq":7252,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385839+0000","last_change":"2026-02-21T22:24:02.430358+0000","last_active":"2026-02-21T22:24:44.385839+0000","last_peered":"2026-02-21T22:24:44.385839+0000","last_clean":"2026-02-21T22:24:44.385839+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:24:44.385839+0000","last_undegraded":"2026-02-21T22:24:44.385839+0000","last_fullsized":"2026-02-21T22:24:44.385839+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:34:07.806548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'7137","reported_seq":7219,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:24:44.385148+0000","last_change":"2026-02-21T22:22:42.379784+0000","last_active":"2026-02-21T22:24:44.385148+0000","last_peered":"2026-02-21T22:24:44.385148+0000","last_clean":"2026-02-21T22:24:44.385148+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:24:44.385148+0000","last_undegraded":"2026-02-21T22:24:44.385148+0000","last_fullsized":"2026-02-21T22:24:44.385148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":0,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:19:44.108645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":1110210,"data_compressed":280101,"data_compressed_allocated":299008,"data_compressed_original":897024,"omap_allocated":0,"internal_metadata":0},"log_size":57716,"ondisk_log_size":57716,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986197,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117015,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226032,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":225443,"kb_avail":94145808,"statfs":{"total":96636764160,"available":96405307392,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":230854284},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":273751,"data_compressed":69066,"data_compressed_allocated":73728,"data_compressed_original":221184,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":258545,"data_compressed":65229,"data_compressed_allocated":69632,"data_compressed_original":208896,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:24:49.770 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.770 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.770 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.771 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.771 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.771 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.771 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.771 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:24:49.771 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T22:25:09.772 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:25:09.967 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:25:09.967 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:25:09.977 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2026-02-21T22:25:08.240478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80118,"ondisk_log_size":80118,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":524256,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":523079,"kb_avail":188219424,"statfs":{"total":193273528320,"available":192736690176,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":535633176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001315"},"pg_stats":[{"pgid":"1.7","version":"47'10589","reported_seq":10878,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.166030+0000","last_change":"2026-02-21T22:24:50.016175+0000","last_active":"2026-02-21T22:25:05.166030+0000","last_peered":"2026-02-21T22:25:05.166030+0000","last_clean":"2026-02-21T22:25:05.166030+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:25:05.166030+0000","last_undegraded":"2026-02-21T22:25:05.166030+0000","last_fullsized":"2026-02-21T22:25:05.166030+0000","mapping_epoch":34,"log_start":"17'500","ondisk_log_start":"17'500","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":3329,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"47'9930","reported_seq":10172,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.278558+0000","last_change":"2026-02-21T22:24:52.855407+0000","last_active":"2026-02-21T22:25:05.278558+0000","last_peered":"2026-02-21T22:25:05.278558+0000","last_clean":"2026-02-21T22:25:05.278558+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:25:05.278558+0000","last_undegraded":"2026-02-21T22:25:05.278558+0000","last_fullsized":"2026-02-21T22:25:05.278558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":2624,"log_size":9930,"log_dups_size":0,"ondisk_log_size":9930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"47'9935","reported_seq":10184,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.248788+0000","last_change":"2026-02-21T22:24:51.891636+0000","last_active":"2026-02-21T22:25:05.248788+0000","last_peered":"2026-02-21T22:25:05.248788+0000","last_clean":"2026-02-21T22:25:05.248788+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:25:05.248788+0000","last_undegraded":"2026-02-21T22:25:05.248788+0000","last_fullsized":"2026-02-21T22:25:05.248788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":2819,"log_size":9935,"log_dups_size":0,"ondisk_log_size":9935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 14s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"47'10304","reported_seq":10598,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.278228+0000","last_change":"2026-02-21T22:24:50.936299+0000","last_active":"2026-02-21T22:25:05.278228+0000","last_peered":"2026-02-21T22:25:05.278228+0000","last_clean":"2026-02-21T22:25:05.278228+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:25:05.278228+0000","last_undegraded":"2026-02-21T22:25:05.278228+0000","last_fullsized":"2026-02-21T22:25:05.278228+0000","mapping_epoch":36,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":3044,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 15s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"47'10174","reported_seq":10430,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.206102+0000","last_change":"2026-02-21T22:24:51.029922+0000","last_active":"2026-02-21T22:25:05.206102+0000","last_peered":"2026-02-21T22:25:05.206102+0000","last_clean":"2026-02-21T22:25:05.206102+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:25:05.206102+0000","last_undegraded":"2026-02-21T22:25:05.206102+0000","last_fullsized":"2026-02-21T22:25:05.206102+0000","mapping_epoch":42,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":2954,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 15s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"47'11323","reported_seq":11653,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.293736+0000","last_change":"2026-02-21T22:24:48.998837+0000","last_active":"2026-02-21T22:25:05.293736+0000","last_peered":"2026-02-21T22:25:05.293736+0000","last_clean":"2026-02-21T22:25:05.293736+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:25:05.293736+0000","last_undegraded":"2026-02-21T22:25:05.293736+0000","last_fullsized":"2026-02-21T22:25:05.293736+0000","mapping_epoch":38,"log_start":"17'1300","ondisk_log_start":"17'1300","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":4094,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 17s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"47'10712","reported_seq":11017,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.278556+0000","last_change":"2026-02-21T22:24:49.952503+0000","last_active":"2026-02-21T22:25:05.278556+0000","last_peered":"2026-02-21T22:25:05.278556+0000","last_clean":"2026-02-21T22:25:05.278556+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:25:05.278556+0000","last_undegraded":"2026-02-21T22:25:05.278556+0000","last_fullsized":"2026-02-21T22:25:05.278556+0000","mapping_epoch":40,"log_start":"17'700","ondisk_log_start":"17'700","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":3539,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"47'11051","reported_seq":11397,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:05.205310+0000","last_change":"2026-02-21T22:24:49.003460+0000","last_active":"2026-02-21T22:25:05.205310+0000","last_peered":"2026-02-21T22:25:05.205310+0000","last_clean":"2026-02-21T22:25:05.205310+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:25:05.205310+0000","last_undegraded":"2026-02-21T22:25:05.205310+0000","last_fullsized":"2026-02-21T22:25:05.205310+0000","mapping_epoch":22,"log_start":"17'1000","ondisk_log_start":"17'1000","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":3914,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 17s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":988562,"data_compressed":249405,"data_compressed_allocated":266240,"data_compressed_original":798720,"omap_allocated":0,"internal_metadata":0},"log_size":80118,"ondisk_log_size":80118,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986201,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117019,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705913,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":262128,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":261539,"kb_avail":94109712,"statfs":{"total":96636764160,"available":96368345088,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":267816588},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":262128,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":261539,"kb_avail":94109712,"statfs":{"total":96636764160,"available":96368345088,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":267816588},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":212927,"data_compressed":53718,"data_compressed_allocated":57344,"data_compressed_original":172032,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":197721,"data_compressed":49881,"data_compressed_allocated":53248,"data_compressed_original":159744,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:25:09.978 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.978 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:09.979 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T22:25:29.980 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:25:30.154 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:25:30.155 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:25:30.163 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":659,"stamp":"2026-02-21T22:25:28.242768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80197,"ondisk_log_size":80197,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":668640,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":667463,"kb_avail":188075040,"statfs":{"total":193273528320,"available":192588840960,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":683482392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"47'13844","reported_seq":14352,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.186638+0000","last_change":"2026-02-21T22:24:50.016175+0000","last_active":"2026-02-21T22:25:25.186638+0000","last_peered":"2026-02-21T22:25:25.186638+0000","last_clean":"2026-02-21T22:25:25.186638+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:25:25.186638+0000","last_undegraded":"2026-02-21T22:25:25.186638+0000","last_fullsized":"2026-02-21T22:25:25.186638+0000","mapping_epoch":34,"log_start":"17'3800","ondisk_log_start":"17'3800","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":6599,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 36s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"47'13515","reported_seq":14005,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.289850+0000","last_change":"2026-02-21T22:24:52.855407+0000","last_active":"2026-02-21T22:25:25.289850+0000","last_peered":"2026-02-21T22:25:25.289850+0000","last_clean":"2026-02-21T22:25:25.289850+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:25:25.289850+0000","last_undegraded":"2026-02-21T22:25:25.289850+0000","last_fullsized":"2026-02-21T22:25:25.289850+0000","mapping_epoch":44,"log_start":"17'3500","ondisk_log_start":"17'3500","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":6209,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 33s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"47'13505","reported_seq":13990,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.297684+0000","last_change":"2026-02-21T22:24:51.891636+0000","last_active":"2026-02-21T22:25:25.297684+0000","last_peered":"2026-02-21T22:25:25.297684+0000","last_clean":"2026-02-21T22:25:25.297684+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:25:25.297684+0000","last_undegraded":"2026-02-21T22:25:25.297684+0000","last_fullsized":"2026-02-21T22:25:25.297684+0000","mapping_epoch":46,"log_start":"17'3500","ondisk_log_start":"17'3500","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":6374,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 34s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"47'13949","reported_seq":14488,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.289811+0000","last_change":"2026-02-21T22:24:50.936299+0000","last_active":"2026-02-21T22:25:25.289811+0000","last_peered":"2026-02-21T22:25:25.289811+0000","last_clean":"2026-02-21T22:25:25.289811+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:25:25.289811+0000","last_undegraded":"2026-02-21T22:25:25.289811+0000","last_fullsized":"2026-02-21T22:25:25.289811+0000","mapping_epoch":36,"log_start":"17'3900","ondisk_log_start":"17'3900","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":6689,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 35s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"47'13519","reported_seq":14000,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.186616+0000","last_change":"2026-02-21T22:24:51.029922+0000","last_active":"2026-02-21T22:25:25.186616+0000","last_peered":"2026-02-21T22:25:25.186616+0000","last_clean":"2026-02-21T22:25:25.186616+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:25:25.186616+0000","last_undegraded":"2026-02-21T22:25:25.186616+0000","last_fullsized":"2026-02-21T22:25:25.186616+0000","mapping_epoch":42,"log_start":"17'3500","ondisk_log_start":"17'3500","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":6299,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 35s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"47'14457","reported_seq":15006,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:22.968465+0000","last_change":"2026-02-21T22:25:22.968465+0000","last_active":"2026-02-21T22:25:22.968465+0000","last_peered":"2026-02-21T22:25:22.968465+0000","last_clean":"2026-02-21T22:25:22.968465+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:25:22.968465+0000","last_undegraded":"2026-02-21T22:25:22.968465+0000","last_fullsized":"2026-02-21T22:25:22.968465+0000","mapping_epoch":38,"log_start":"17'4400","ondisk_log_start":"17'4400","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14457","last_scrub_stamp":"2026-02-21T22:25:22.968423+0000","last_deep_scrub":"47'14457","last_deep_scrub_stamp":"2026-02-21T22:25:22.968423+0000","last_clean_scrub_stamp":"2026-02-21T22:25:22.968423+0000","objects_scrubbed":7228,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":34,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:56:45.587900+0000","scrub_duration":33969,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"47'14207","reported_seq":14761,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.245672+0000","last_change":"2026-02-21T22:24:49.952503+0000","last_active":"2026-02-21T22:25:25.245672+0000","last_peered":"2026-02-21T22:25:25.245672+0000","last_clean":"2026-02-21T22:25:25.245672+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:25:25.245672+0000","last_undegraded":"2026-02-21T22:25:25.245672+0000","last_fullsized":"2026-02-21T22:25:25.245672+0000","mapping_epoch":40,"log_start":"17'4200","ondisk_log_start":"17'4200","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":7034,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 36s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"47'14201","reported_seq":14760,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-21T22:25:25.150109+0000","last_change":"2026-02-21T22:24:49.003460+0000","last_active":"2026-02-21T22:25:25.150109+0000","last_peered":"2026-02-21T22:25:25.150109+0000","last_clean":"2026-02-21T22:25:25.150109+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:25:25.150109+0000","last_undegraded":"2026-02-21T22:25:25.150109+0000","last_fullsized":"2026-02-21T22:25:25.150109+0000","mapping_epoch":22,"log_start":"17'4200","ondisk_log_start":"17'4200","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:05:04.195147+0000","last_clean_scrub_stamp":"2026-02-21T22:05:04.195147+0000","objects_scrubbed":7079,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":942944,"data_compressed":237894,"data_compressed_allocated":253952,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},"log_size":80197,"ondisk_log_size":80197,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986205,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117023,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705917,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334320,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":333731,"kb_avail":94037520,"statfs":{"total":96636764160,"available":96294420480,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":341741196},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334320,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":333731,"kb_avail":94037520,"statfs":{"total":96636764160,"available":96294420480,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":341741196},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":182515,"data_compressed":46044,"data_compressed_allocated":49152,"data_compressed_original":147456,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":182515,"data_compressed":46044,"data_compressed_allocated":49152,"data_compressed_original":147456,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:25:30.164 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.164 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.165 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.165 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.165 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.165 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.165 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T22:05:04.195147+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=5, tm_sec=4, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=24, tm_sec=47, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:25:30.165 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T22:25:50.166 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:25:50.341 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-21T22:25:50.341 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-21T22:25:50.350 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":670,"stamp":"2026-02-21T22:25:50.245215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80331,"ondisk_log_size":80331,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":668640,"kb_used_data":1088,"kb_used_omap":56,"kb_used_meta":667463,"kb_avail":188075040,"statfs":{"total":193273528320,"available":192588840960,"internally_reserved":0,"allocated":1114112,"data_stored":870794,"data_compressed":145806,"data_compressed_allocated":155648,"data_compressed_original":466944,"omap_allocated":58088,"internal_metadata":683482392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001278"},"pg_stats":[{"pgid":"1.7","version":"47'14520","reported_seq":15077,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:28.031193+0000","last_change":"2026-02-21T22:25:28.031193+0000","last_active":"2026-02-21T22:25:28.031193+0000","last_peered":"2026-02-21T22:25:28.031193+0000","last_clean":"2026-02-21T22:25:28.031193+0000","last_became_active":"2026-02-21T22:23:15.364917+0000","last_became_peered":"2026-02-21T22:23:15.364917+0000","last_unstale":"2026-02-21T22:25:28.031193+0000","last_undegraded":"2026-02-21T22:25:28.031193+0000","last_fullsized":"2026-02-21T22:25:28.031193+0000","mapping_epoch":34,"log_start":"17'4500","ondisk_log_start":"17'4500","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14520","last_scrub_stamp":"2026-02-21T22:25:28.031150+0000","last_deep_scrub":"47'14520","last_deep_scrub_stamp":"2026-02-21T22:25:28.031150+0000","last_clean_scrub_stamp":"2026-02-21T22:25:28.031150+0000","objects_scrubbed":7260,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:10:11.394364+0000","scrub_duration":38013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"47'14612","reported_seq":15180,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:28.544650+0000","last_change":"2026-02-21T22:25:28.544650+0000","last_active":"2026-02-21T22:25:28.544650+0000","last_peered":"2026-02-21T22:25:28.544650+0000","last_clean":"2026-02-21T22:25:28.544650+0000","last_became_active":"2026-02-21T22:24:29.366238+0000","last_became_peered":"2026-02-21T22:24:29.366238+0000","last_unstale":"2026-02-21T22:25:28.544650+0000","last_undegraded":"2026-02-21T22:25:28.544650+0000","last_fullsized":"2026-02-21T22:25:28.544650+0000","mapping_epoch":44,"log_start":"17'4600","ondisk_log_start":"17'4600","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14612","last_scrub_stamp":"2026-02-21T22:25:28.544627+0000","last_deep_scrub":"47'14612","last_deep_scrub_stamp":"2026-02-21T22:25:28.544627+0000","last_clean_scrub_stamp":"2026-02-21T22:25:28.544627+0000","objects_scrubbed":7306,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:58:56.852272+0000","scrub_duration":35689,"objects_trimmed":0,"snaptrim_duration":0.00022109300000000001,"stat_sum":{"num_bytes":68126,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5654,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"47'14262","reported_seq":14805,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:28.160879+0000","last_change":"2026-02-21T22:25:28.160879+0000","last_active":"2026-02-21T22:25:28.160879+0000","last_peered":"2026-02-21T22:25:28.160879+0000","last_clean":"2026-02-21T22:25:28.160879+0000","last_became_active":"2026-02-21T22:24:44.386068+0000","last_became_peered":"2026-02-21T22:24:44.386068+0000","last_unstale":"2026-02-21T22:25:28.160879+0000","last_undegraded":"2026-02-21T22:25:28.160879+0000","last_fullsized":"2026-02-21T22:25:28.160879+0000","mapping_epoch":46,"log_start":"17'4200","ondisk_log_start":"17'4200","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14262","last_scrub_stamp":"2026-02-21T22:25:28.160841+0000","last_deep_scrub":"47'14262","last_deep_scrub_stamp":"2026-02-21T22:25:28.160841+0000","last_clean_scrub_stamp":"2026-02-21T22:25:28.160841+0000","objects_scrubbed":7131,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":37,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:48:31.331735+0000","scrub_duration":36257,"objects_trimmed":0,"snaptrim_duration":0.00021177500000000001,"stat_sum":{"num_bytes":37564,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":7159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7131,"num_bytes_recovered":30436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"47'14520","reported_seq":15102,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:27.655922+0000","last_change":"2026-02-21T22:25:27.655922+0000","last_active":"2026-02-21T22:25:27.655922+0000","last_peered":"2026-02-21T22:25:27.655922+0000","last_clean":"2026-02-21T22:25:27.655922+0000","last_became_active":"2026-02-21T22:23:30.858644+0000","last_became_peered":"2026-02-21T22:23:30.858644+0000","last_unstale":"2026-02-21T22:25:27.655922+0000","last_undegraded":"2026-02-21T22:25:27.655922+0000","last_fullsized":"2026-02-21T22:25:27.655922+0000","mapping_epoch":36,"log_start":"17'4500","ondisk_log_start":"17'4500","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14520","last_scrub_stamp":"2026-02-21T22:25:27.655882+0000","last_deep_scrub":"47'14520","last_deep_scrub_stamp":"2026-02-21T22:25:27.655882+0000","last_clean_scrub_stamp":"2026-02-21T22:25:27.655882+0000","objects_scrubbed":7260,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":37,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:53:38.863762+0000","scrub_duration":36717,"objects_trimmed":0,"snaptrim_duration":0.00023290499999999999,"stat_sum":{"num_bytes":68080,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7260,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"47'14440","reported_seq":14987,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:28.407855+0000","last_change":"2026-02-21T22:25:28.407855+0000","last_active":"2026-02-21T22:25:28.407855+0000","last_peered":"2026-02-21T22:25:28.407855+0000","last_clean":"2026-02-21T22:25:28.407855+0000","last_became_active":"2026-02-21T22:24:18.297443+0000","last_became_peered":"2026-02-21T22:24:18.297443+0000","last_unstale":"2026-02-21T22:25:28.407855+0000","last_undegraded":"2026-02-21T22:25:28.407855+0000","last_fullsized":"2026-02-21T22:25:28.407855+0000","mapping_epoch":42,"log_start":"17'4400","ondisk_log_start":"17'4400","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14440","last_scrub_stamp":"2026-02-21T22:25:28.407822+0000","last_deep_scrub":"47'14440","last_deep_scrub_stamp":"2026-02-21T22:25:28.407822+0000","last_clean_scrub_stamp":"2026-02-21T22:25:28.407822+0000","objects_scrubbed":7220,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:47:57.676790+0000","scrub_duration":37355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":7247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":30412,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"47'14457","reported_seq":15006,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:22.968465+0000","last_change":"2026-02-21T22:25:22.968465+0000","last_active":"2026-02-21T22:25:22.968465+0000","last_peered":"2026-02-21T22:25:22.968465+0000","last_clean":"2026-02-21T22:25:22.968465+0000","last_became_active":"2026-02-21T22:23:47.297658+0000","last_became_peered":"2026-02-21T22:23:47.297658+0000","last_unstale":"2026-02-21T22:25:22.968465+0000","last_undegraded":"2026-02-21T22:25:22.968465+0000","last_fullsized":"2026-02-21T22:25:22.968465+0000","mapping_epoch":38,"log_start":"17'4400","ondisk_log_start":"17'4400","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14457","last_scrub_stamp":"2026-02-21T22:25:22.968423+0000","last_deep_scrub":"47'14457","last_deep_scrub_stamp":"2026-02-21T22:25:22.968423+0000","last_clean_scrub_stamp":"2026-02-21T22:25:22.968423+0000","objects_scrubbed":7228,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":34,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:56:45.587900+0000","scrub_duration":33969,"objects_trimmed":0,"snaptrim_duration":0.00025430500000000003,"stat_sum":{"num_bytes":37656,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":7257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7229,"num_bytes_recovered":30450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"47'14346","reported_seq":14914,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:26.047084+0000","last_change":"2026-02-21T22:25:26.047084+0000","last_active":"2026-02-21T22:25:26.047084+0000","last_peered":"2026-02-21T22:25:26.047084+0000","last_clean":"2026-02-21T22:25:26.047084+0000","last_became_active":"2026-02-21T22:24:02.427356+0000","last_became_peered":"2026-02-21T22:24:02.427356+0000","last_unstale":"2026-02-21T22:25:26.047084+0000","last_undegraded":"2026-02-21T22:25:26.047084+0000","last_fullsized":"2026-02-21T22:25:26.047084+0000","mapping_epoch":40,"log_start":"17'4300","ondisk_log_start":"17'4300","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14346","last_scrub_stamp":"2026-02-21T22:25:26.047037+0000","last_deep_scrub":"47'14346","last_deep_scrub_stamp":"2026-02-21T22:25:26.047037+0000","last_clean_scrub_stamp":"2026-02-21T22:25:26.047037+0000","objects_scrubbed":7173,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":37,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:34:00.245706+0000","scrub_duration":36095,"objects_trimmed":0,"snaptrim_duration":0.000216715,"stat_sum":{"num_bytes":22378,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":7187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7173,"num_bytes_recovered":15206,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"47'14274","reported_seq":14841,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-21T22:25:25.572428+0000","last_change":"2026-02-21T22:25:25.572428+0000","last_active":"2026-02-21T22:25:25.572428+0000","last_peered":"2026-02-21T22:25:25.572428+0000","last_clean":"2026-02-21T22:25:25.572428+0000","last_became_active":"2026-02-21T22:22:42.376315+0000","last_became_peered":"2026-02-21T22:22:42.376315+0000","last_unstale":"2026-02-21T22:25:25.572428+0000","last_undegraded":"2026-02-21T22:25:25.572428+0000","last_fullsized":"2026-02-21T22:25:25.572428+0000","mapping_epoch":22,"log_start":"17'4200","ondisk_log_start":"17'4200","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14274","last_scrub_stamp":"2026-02-21T22:25:25.572390+0000","last_deep_scrub":"47'14274","last_deep_scrub_stamp":"2026-02-21T22:25:25.572390+0000","last_clean_scrub_stamp":"2026-02-21T22:25:25.572390+0000","objects_scrubbed":7137,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":37,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:39:02.220427+0000","scrub_duration":36568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67957,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7137,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7137,"num_bytes_recovered":60824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":346650,"num_objects":57715,"num_object_clones":0,"num_object_copies":115430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57715,"num_write_kb":57981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56065,"num_bytes_recovered":288976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":897326,"data_compressed":226383,"data_compressed_allocated":241664,"data_compressed_original":724992,"omap_allocated":0,"internal_metadata":0},"log_size":80331,"ondisk_log_size":80331,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":133143986209,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":115964117027,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705922,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334320,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":333731,"kb_avail":94037520,"statfs":{"total":96636764160,"available":96294420480,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":341741196},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705920,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334320,"kb_used_data":544,"kb_used_omap":28,"kb_used_meta":333731,"kb_avail":94037520,"statfs":{"total":96636764160,"available":96294420480,"internally_reserved":0,"allocated":557056,"data_stored":435397,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":29044,"internal_metadata":341741196},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":288957,"data_compressed":72903,"data_compressed_allocated":77824,"data_compressed_original":233472,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":167309,"data_compressed":42207,"data_compressed_allocated":45056,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":152103,"data_compressed":38370,"data_compressed_allocated":40960,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:25:50.351 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T22:25:50.536 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T22:25:50.536 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T22:25:50.537 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T22:25:50.537 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.587 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T22:25:50.587 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T22:25:50.587 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.642 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T22:25:50.643 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T22:25:50.643 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.684 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T22:25:50.685 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-21T22:25:50.685 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.728 INFO:tasks.ceph.osd.3:Stopped 2026-02-21T22:25:50.728 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T22:25:50.729 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T22:25:50.729 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.745 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T22:25:50.746 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T22:25:50.746 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T22:25:50.746 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.797 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T22:25:50.797 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:25:50.797 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.846 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:25:50.847 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-21T22:25:50.847 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:50.896 INFO:tasks.ceph.mon.c:Stopped 2026-02-21T22:25:50.896 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T22:25:50.897 DEBUG:teuthology.orchestra.run.trial060:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v 'but it is still running' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(OBJECT_' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2026-02-21T22:25:50.963 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:25:50.963 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T22:25:51.061 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:25:51.061 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T22:25:51.110 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:25:51.110 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T22:25:51.150 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:25:51.150 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-21T22:25:51.230 INFO:tasks.ceph:Archiving mon data... 2026-02-21T22:25:51.231 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078/data/mon.a.tgz 2026-02-21T22:25:51.231 DEBUG:teuthology.orchestra.run.trial060:> mktemp 2026-02-21T22:25:51.287 INFO:teuthology.orchestra.run.trial060.stdout:/tmp/tmp.r40Kna7nLg 2026-02-21T22:25:51.288 DEBUG:teuthology.orchestra.run.trial060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.r40Kna7nLg 2026-02-21T22:25:51.385 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0666 /tmp/tmp.r40Kna7nLg 2026-02-21T22:25:51.466 DEBUG:teuthology.orchestra.remote:trial060:/tmp/tmp.r40Kna7nLg is 282KB 2026-02-21T22:25:51.561 DEBUG:teuthology.orchestra.run.trial060:> rm -fr /tmp/tmp.r40Kna7nLg 2026-02-21T22:25:51.579 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078/data/mon.b.tgz 2026-02-21T22:25:51.580 DEBUG:teuthology.orchestra.run.trial060:> mktemp 2026-02-21T22:25:51.635 INFO:teuthology.orchestra.run.trial060.stdout:/tmp/tmp.d0F7gu7POe 2026-02-21T22:25:51.635 DEBUG:teuthology.orchestra.run.trial060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.d0F7gu7POe 2026-02-21T22:25:51.735 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0666 /tmp/tmp.d0F7gu7POe 2026-02-21T22:25:51.819 DEBUG:teuthology.orchestra.remote:trial060:/tmp/tmp.d0F7gu7POe is 299KB 2026-02-21T22:25:51.913 DEBUG:teuthology.orchestra.run.trial060:> rm -fr /tmp/tmp.d0F7gu7POe 2026-02-21T22:25:51.930 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078/data/mon.c.tgz 2026-02-21T22:25:51.931 DEBUG:teuthology.orchestra.run.trial060:> mktemp 2026-02-21T22:25:51.987 INFO:teuthology.orchestra.run.trial060.stdout:/tmp/tmp.W5rxyhYfQ0 2026-02-21T22:25:51.987 DEBUG:teuthology.orchestra.run.trial060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.W5rxyhYfQ0 2026-02-21T22:25:52.085 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0666 /tmp/tmp.W5rxyhYfQ0 2026-02-21T22:25:52.165 DEBUG:teuthology.orchestra.remote:trial060:/tmp/tmp.W5rxyhYfQ0 is 299KB 2026-02-21T22:25:52.258 DEBUG:teuthology.orchestra.run.trial060:> rm -fr /tmp/tmp.W5rxyhYfQ0 2026-02-21T22:25:52.275 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T22:25:52.275 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T22:25:52.395 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T22:25:52.395 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T22:25:52.399 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078/remote/trial060/crash 2026-02-21T22:25:52.401 DEBUG:teuthology.orchestra.run.trial060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T22:25:52.460 INFO:tasks.ceph:Compressing logs... 2026-02-21T22:25:52.460 DEBUG:teuthology.orchestra.run.trial060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T22:25:52.577 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70083.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97010.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70083.log.gz 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54239.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97010.log.gz 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102365.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86276.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54239.log.gz 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102365.log.gzgzip 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52473.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71553.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86276.log.gz 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75622.log 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52473.log.gz 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71553.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71553.log.gz -5 2026-02-21T22:25:52.578 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72975.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75622.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109999.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81470.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72975.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110832.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109999.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81470.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110587.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110832.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86766.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60671.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110587.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86766.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52817.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68857.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60671.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64592.log 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52817.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68857.log.gz 2026-02-21T22:25:52.579 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67485.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74592.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64592.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65523.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67485.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74592.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.74592.log.gz --verbose 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: -- /var/log/ceph/ceph-client.admin.71406.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55268.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65523.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71406.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79349.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82600.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55268.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60868.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79349.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51247.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82600.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93627.log 0.0% 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60868.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100637.log 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51247.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93627.log.gz 2026-02-21T22:25:52.580 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65376.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5/var/log/ceph/ceph-client.admin.100637.log: --verbose -- /var/log/ceph/ceph-client.admin.102071.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100637.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65376.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82943.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88726.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102071.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82943.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62093.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109019.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88726.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62093.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85688.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101973.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109019.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85688.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60034.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73710.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101973.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60034.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58858.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89265.log 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73710.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58858.log.gz 2026-02-21T22:25:52.581 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60475.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98970.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89265.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60475.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103863.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92597.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98970.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103863.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101189.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90098.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92597.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101189.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83727.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50012.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90098.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83727.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63808.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83090.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50012.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63808.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87256.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83090.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108970.log 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87256.log.gz 2026-02-21T22:25:52.582 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51639.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108970.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90049.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51639.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58061.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90049.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65131.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58061.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87354.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108382.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65131.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87354.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111175.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85296.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108382.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61848.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111175.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109068.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85296.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61848.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107451.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111077.log 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109068.log.gz 2026-02-21T22:25:52.583 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107451.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.107451.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr: /var/log/ceph/ceph-client.admin.66897.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111077.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85786.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99559.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66897.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85786.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105011.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91911.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99559.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.105011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105011.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77781.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112254.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91911.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77781.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89951.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59887.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112254.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89951.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80931.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59887.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82404.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59642.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80931.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82404.log.gz 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108823.log 2026-02-21T22:25:52.584 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59642.log: -- /var/log/ceph/ceph-client.admin.65819.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59642.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67877.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108823.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65819.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88873.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88236.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67877.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88873.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69838.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88236.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96616.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63759.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69838.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96616.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112009.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59201.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63759.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112009.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108235.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110979.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59201.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108235.log.gz 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59936.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69298.log 2026-02-21T22:25:52.585 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110979.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59936.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62289.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54386.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69298.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62289.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110440.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71896.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54386.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110440.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68759.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71896.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58809.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68759.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98774.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58809.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97059.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98774.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72141.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97059.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108676.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72141.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74249.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108676.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55856.log 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74249.log.gz 2026-02-21T22:25:52.586 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58698.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55856.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77046.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58698.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54925.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77046.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91421.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54925.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65327.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91421.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85492.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65327.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96126.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85492.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88824.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96126.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86619.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88824.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81718.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86619.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71063.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81718.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72435.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71063.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109901.log 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72435.log.gz 2026-02-21T22:25:52.587 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95538.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109901.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76605.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95538.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48428.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76605.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89461.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48428.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78957.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89461.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100588.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78957.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88628.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100588.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79741.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93480.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88628.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79741.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74935.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99510.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93480.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74935.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83629.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99510.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92891.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83629.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54189.log 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92891.log.gz 2026-02-21T22:25:52.588 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63073.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54189.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63073.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53160.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70573.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53160.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70573.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48403.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75769.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48403.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75769.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69592.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44652.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69592.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86815.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72779.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44652.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86815.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111323.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51787.log 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72779.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111323.log.gz 2026-02-21T22:25:52.589 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104646.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71112.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51787.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104646.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103442.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79055.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71112.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103442.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98921.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90931.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79055.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98921.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100000.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87648.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90931.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100000.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64151.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107745.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87648.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64151.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80686.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71602.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107745.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80686.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102218.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71602.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71602.log.gz -5 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64200.log 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102218.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2026-02-21T22:25:52.590 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71259.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64200.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74837.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71259.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69494.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79545.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74837.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69494.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97500.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79545.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69887.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97500.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59593.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69887.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80343.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59593.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89216.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58159.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80343.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89216.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84070.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58159.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79790.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84070.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109117.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79790.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109117.log.gz 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98578.log 2026-02-21T22:25:52.591 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109264.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98578.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49663.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109264.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65229.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49663.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68171.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65229.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51051.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68171.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57032.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51051.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92009.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57032.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100797.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92009.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50441.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100797.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74396.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83825.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50441.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74396.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98480.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83825.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94264.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98480.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103530.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94264.log.gz 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92842.log 2026-02-21T22:25:52.592 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103530.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95440.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92842.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95440.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112597.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97353.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112597.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99412.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97353.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51541.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99412.log.gz 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93088.log 2026-02-21T22:25:52.593 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51541.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69102.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93088.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102414.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69102.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76703.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102414.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94068.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76703.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82894.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94068.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73171.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82894.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55513.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73171.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49856.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104744.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55513.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49856.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54288.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104744.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72337.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54288.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70524.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72337.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109313.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70524.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84512.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109313.log.gz 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60083.log 2026-02-21T22:25:52.603 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84512.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89559.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60083.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70769.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89559.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83188.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70769.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89020.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83188.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107794.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89020.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73367.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107794.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87746.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73367.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112794.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87746.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68808.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112794.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50298.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68808.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65033.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50298.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65033.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100539.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88579.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100539.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82159.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88579.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107941.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82159.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70230.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107941.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91078.log 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70230.log.gz 2026-02-21T22:25:52.604 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61505.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91078.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88775.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61505.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81323.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88775.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92744.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81323.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66113.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92744.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66113.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103090.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92352.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103090.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110538.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84414.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92352.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110538.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93872.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97206.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84414.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93872.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92303.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89412.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97206.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92303.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103002.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93333.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89412.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103002.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82110.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103178.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93333.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82110.log.gz 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85149.log 2026-02-21T22:25:52.605 INFO:teuthology.orchestra.run.trial060.stderr:gzip/var/log/ceph/ceph-client.admin.103178.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.77830.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103178.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85149.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66456.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77830.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61064.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66456.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63955.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61064.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88187.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63955.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100049.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88187.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90147.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100049.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74053.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90147.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83237.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74053.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111715.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83237.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111715.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110146.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101826.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110146.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48640.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101826.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87893.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48640.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72190.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87893.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49454.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72190.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64347.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49454.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63220.log 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64347.log.gz 2026-02-21T22:25:52.606 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111911.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63220.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84659.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61750.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111911.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84659.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51983.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61750.log: /var/log/ceph/ceph-client.admin.48768.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61750.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51983.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97794.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48768.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53944.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97794.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59446.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53944.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61015.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59446.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76997.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61015.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76997.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102870.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53209.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102870.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72877.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53209.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53405.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72877.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93529.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82012.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53405.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53454.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93529.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82012.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78075.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67779.log 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53454.log.gz 2026-02-21T22:25:52.607 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78075.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86227.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67779.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74494.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86227.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109411.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74494.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61603.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109411.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86864.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61603.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86864.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50297.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111028.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50297.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87991.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111028.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80098.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87991.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66407.log 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80098.log.gz 2026-02-21T22:25:52.608 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79496.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73563.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66407.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79496.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73563.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103867.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54729.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103867.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88530.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54729.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58355.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91568.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88530.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58355.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97941.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111666.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91568.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75720.log 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97941.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111666.log.gz 2026-02-21T22:25:52.611 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80196.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79937.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75720.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80196.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110930.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79937.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110342.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69249.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110930.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110342.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55023.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69249.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96518.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip/var/log/ceph/ceph-client.admin.55023.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73759.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55023.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96518.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74004.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89706.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73759.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74004.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81225.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59348.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89706.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81225.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75328.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59348.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90343.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75328.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52424.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90343.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69053.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52424.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53552.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69053.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65966.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53552.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77879.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65966.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110489.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77879.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102758.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110489.log.gz 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70671.log 2026-02-21T22:25:52.612 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102758.log.gz 2026-02-21T22:25:52.613 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58306.log 2026-02-21T22:25:52.613 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70671.log.gz 2026-02-21T22:25:52.613 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84119.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58306.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112695.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84119.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90392.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112695.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94950.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90392.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85198.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94950.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63514.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85198.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86080.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63514.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99020.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86080.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76308.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99020.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56542.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80882.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76308.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56542.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83041.log 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80882.log.gz 2026-02-21T22:25:52.614 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69936.log 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83041.log.gz 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69445.log 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69936.log.gz 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72386.log 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69445.log.gz 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103486.log 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72386.log.gz 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82453.log 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103486.log.gz 2026-02-21T22:25:52.615 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67044.log 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82453.log.gz 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74445.log 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67044.log.gz 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64494.log 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74445.log.gz 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51885.log 2026-02-21T22:25:52.616 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64494.log.gz 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60132.log 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83678.log 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51885.log.gz 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60132.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.60132.log.gz /var/log/ceph/ceph-client.admin.93578.log 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83678.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.71014.log -- replaced with /var/log/ceph/ceph-client.admin.83678.log.gz 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.617 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93578.log.gz 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91127.log 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98627.log 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71014.log.gz 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91127.log.gz 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84168.log 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98627.log.gz 2026-02-21T22:25:52.618 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68220.log 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84168.log.gz 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57228.log 0.0% 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68220.log.gz 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85541.log 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108186.log 2026-02-21T22:25:52.619 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57228.log.gz 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85541.log.gz 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78516.log 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63857.log 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108186.log.gz 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78516.log.gz 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84217.log 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63857.log.gz 2026-02-21T22:25:52.620 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75426.log 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84217.log.gz 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96322.log 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75426.log.gz 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102781.log 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96322.log.gz 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108725.log 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102781.log.gz 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103862.log 2026-02-21T22:25:52.621 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108725.log.gz 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103354.log 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103862.log.gz 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50538.log 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103354.log.gz 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92401.log 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50538.log.gz 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58747.log 2026-02-21T22:25:52.622 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67828.log 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92401.log.gz 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58747.log.gz 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103266.log 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51934.log 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67828.log.gz 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103266.log.gz 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107647.log 2026-02-21T22:25:52.623 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51934.log.gz 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78124.log 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107647.log.gz 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100895.log 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78124.log.gz 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75279.log 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100895.log.gz 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112744.log 2026-02-21T22:25:52.624 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75279.log.gz 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72043.log 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112744.log.gz 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96567.log 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72043.log.gz 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96765.log 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96567.log.gz 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78614.log 2026-02-21T22:25:52.625 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55905.log 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96765.log.gz 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78369.log 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78614.log.gz 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108872.log 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55905.log.gz 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50440.log 2026-02-21T22:25:52.626 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78369.log.gz 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108872.log.gz 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85051.log 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50440.log.gz 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68710.log 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85051.log.gz 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70818.log 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68710.log.gz 2026-02-21T22:25:52.627 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108627.log 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70818.log.gz 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92990.log 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108627.log.gz 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84021.log 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92990.log.gz 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63318.log 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84021.log.gz 2026-02-21T22:25:52.628 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96420.log 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63318.log.gz 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86031.log 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96420.log.gz 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82551.log 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86031.log.gz 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111764.log 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82551.log.gz 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69691.log 2026-02-21T22:25:52.629 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111764.log.gz 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50855.log 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69691.log.gz 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59250.log 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50855.log.gz 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100098.log 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59250.log.gz 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89118.log 2026-02-21T22:25:52.630 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100098.log.gz 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81669.log 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89118.log.gz 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99853.log 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81669.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.53307.log -- replaced with /var/log/ceph/ceph-client.admin.81669.log.gz 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99853.log.gz 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112156.log 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53307.log.gz 2026-02-21T22:25:52.631 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86374.log 2026-02-21T22:25:52.632 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112156.log.gz 2026-02-21T22:25:52.632 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91176.log 2026-02-21T22:25:52.632 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86374.log.gz 2026-02-21T22:25:52.632 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87942.log 2026-02-21T22:25:52.632 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91176.log.gz 2026-02-21T22:25:52.632 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84610.log 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87942.log.gz 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112843.log 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87403.log 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84610.log.gz 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112843.log.gz 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51688.log 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66015.log 2026-02-21T22:25:52.633 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87403.log.gz 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62681.log 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51688.log.gz 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97108.log 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66015.log.gz 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102958.log 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62681.log.gz 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97108.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.97108.log.gz /var/log/ceph/ceph-client.admin.62387.log 2026-02-21T22:25:52.634 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102958.log.gz 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62387.log.gz 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79643.log 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98823.log 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz 2026-02-21T22:25:52.635 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79643.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.93970.log -- replaced with /var/log/ceph/ceph-client.admin.79643.log.gz 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98823.log.gz 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92205.log 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93970.log.gz 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64837.log 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92205.log.gz 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110244.log 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64837.log.gz 2026-02-21T22:25:52.636 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83923.log 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110244.log.gz 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59397.log 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83923.log.gz 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82502.log 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59397.log.gz 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57571.log 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82502.log.gz 2026-02-21T22:25:52.637 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66358.log 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57571.log.gz 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111470.log 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66358.log.gz 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111519.log 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111470.log.gz 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52277.log 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111519.log.gz 2026-02-21T22:25:52.638 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109215.log 2026-02-21T22:25:52.639 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52277.log.gz 2026-02-21T22:25:52.639 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107549.log 2026-02-21T22:25:52.639 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109215.log.gz 2026-02-21T22:25:52.639 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65180.log 2026-02-21T22:25:52.639 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107549.log.gz 2026-02-21T22:25:52.639 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107108.log 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65180.log.gz 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53258.log 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107108.log.gz 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80735.log 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53258.log.gz 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80980.log 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80735.log.gz 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76948.log 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80980.log.gz 2026-02-21T22:25:52.640 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75180.log 2026-02-21T22:25:52.641 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76948.log.gz 2026-02-21T22:25:52.641 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55415.log 2026-02-21T22:25:52.641 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75180.log.gz 2026-02-21T22:25:52.641 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63367.log 2026-02-21T22:25:52.641 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89657.log 2026-02-21T22:25:52.641 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55415.log.gz 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63367.log.gz 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87795.log 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89657.log.gz 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55317.log 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87795.log.gz 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77193.log 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55317.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.55317.log.gz -5 2026-02-21T22:25:52.642 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77487.log 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77193.log.gz 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50296.log 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54582.log 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77487.log.gz 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50296.log.gz 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73612.log 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54582.log.gz 2026-02-21T22:25:52.643 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70965.log 2026-02-21T22:25:52.644 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2026-02-21T22:25:52.644 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73612.log.gz 2026-02-21T22:25:52.644 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70965.log.gz 2026-02-21T22:25:52.644 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56444.log 2026-02-21T22:25:52.644 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95195.log 2026-02-21T22:25:52.644 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58453.log 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56444.log.gz 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95195.log.gz 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48542.log 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr:gzip/var/log/ceph/ceph-client.admin.58453.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99608.log 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58453.log.gz 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48542.log.gz 2026-02-21T22:25:52.645 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76210.log 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104645.log 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99608.log.gz 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76210.log.gz 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55807.log 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104645.log.gz 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100343.log 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55807.log.gz 2026-02-21T22:25:52.646 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61456.log 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100343.log.gz 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82257.log 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61456.log.gz 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64886.log 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82257.log.gz 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68514.log 2026-02-21T22:25:52.647 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64886.log.gz 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97892.log 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60720.log 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68514.log.gz 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104009.log 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97892.log.gz 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60720.log.gz 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64739.log 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101287.log 2026-02-21T22:25:52.648 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104009.log.gz 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60279.log 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64739.log.gz 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78761.log 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101287.log.gz 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76850.log 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60279.log.gz 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91372.log 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78761.log.gz 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69985.log 2026-02-21T22:25:52.649 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76850.log.gz 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76161.log 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91372.log.gz 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79006.log 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69985.log.gz 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96814.log 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76161.log.gz 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62534.log 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79006.log.gz 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55611.log 2026-02-21T22:25:52.650 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96814.log.gz 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63906.log 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62534.log.gz 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73416.log 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55611.log.gz 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55219.log 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63906.log.gz 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81767.log 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73416.log.gz 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57375.log 2026-02-21T22:25:52.651 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55219.log.gz 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61799.log 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81767.log.gz 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63465.log 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57375.log.gz 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94901.log 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61799.log.gz 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93774.log 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63465.log.gz 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99657.log 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94901.log.gz 2026-02-21T22:25:52.652 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111273.log 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93774.log.gz 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87158.log 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99657.log.gz 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96666.log 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111273.log.gz 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73269.log 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87158.log.gz 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95881.log 2026-02-21T22:25:52.653 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96666.log.gz 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59495.log 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73269.log.gz 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59740.log 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95881.log.gz 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93382.log 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59495.log.gz 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102659.log 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59740.log.gz 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51443.log 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93382.log.gz 2026-02-21T22:25:52.654 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95489.log 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102659.log.gz 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108431.log 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51443.log.gz 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86570.log 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95489.log.gz 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79153.log 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108431.log.gz 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95930.log 2026-02-21T22:25:52.655 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86570.log.gz 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79447.log 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79153.log.gz 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95930.log.gz 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79447.log.gz 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75229.log 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71210.log 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87305.log 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75229.log: /var/log/ceph/ceph-client.admin.71210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75229.log.gz 2026-02-21T22:25:52.656 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57963.log 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71210.log.gz 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65082.log 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87305.log.gz 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64788.log 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57963.log.gz 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88334.log 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65082.log.gz 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68955.log 2026-02-21T22:25:52.657 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64788.log.gz 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86913.log 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88334.log.gz 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77144.log 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68955.log.gz 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50169.log 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86913.log.gz 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52768.log 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77144.log.gz 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96469.log 2026-02-21T22:25:52.658 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50169.log.gz 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54778.log 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52768.log.gz 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80147.log 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96469.log.gz 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89608.log 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54778.log.gz 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99951.log 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80147.log.gz 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44757.log 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89608.log.gz 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69347.log 2026-02-21T22:25:52.659 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99951.log.gz 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52669.log 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44757.log.gz 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56640.log 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69347.log.gz 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103398.log 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52669.log.gz 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110881.log 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56640.log.gz 2026-02-21T22:25:52.660 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75671.log 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103398.log.gz 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89167.log 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110881.log.gz 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94558.log 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75671.log.gz 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68416.log 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89167.log.gz 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87207.log 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94558.log.gz 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68416.log.gz 2026-02-21T22:25:52.661 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104842.log 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr:gzip/var/log/ceph/ceph-client.admin.87207.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94509.log 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87207.log.gz 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74200.log 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104842.log.gz 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71847.log 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94509.log.gz 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56346.log 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74200.log.gz 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74543.log 2026-02-21T22:25:52.662 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71847.log.gz 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112939.log 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56346.log.gz 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87550.log 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74543.log.gz 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56836.log 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112939.log.gz 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70720.log 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87550.log.gz 2026-02-21T22:25:52.663 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64690.log 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56836.log.gz 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79839.log 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70720.log.gz 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83433.log 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64690.log.gz 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56591.log 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79839.log.gz 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94019.log 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83433.log.gz 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57865.log 2026-02-21T22:25:52.664 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56591.log.gz 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90294.log 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94019.log.gz 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55072.log 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57865.log.gz 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71945.log 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90294.log.gz 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76556.log 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55072.log.gz 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71945.log.gz 2026-02-21T22:25:52.665 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64984.log 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103688.log 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76556.log.gz 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64984.log.gz 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51836.log 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92793.log 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103688.log.gz 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94607.log 2026-02-21T22:25:52.666 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51836.log.gz 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83776.log 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92793.log.gz 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94607.log.gz 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103866.log 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82698.log 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83776.log.gz 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82649.log 2026-02-21T22:25:52.667 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103866.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90784.log 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82698.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109705.log 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82649.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53356.log 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90784.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60622.log 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109705.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89804.log 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53356.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60622.log.gz 2026-02-21T22:25:52.668 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96863.log 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89804.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89804.log.gz -5 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57424.log 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74641.log 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96863.log.gz 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70426.log 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57424.log.gz 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72681.log 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74641.log.gz 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70426.log.gz 2026-02-21T22:25:52.669 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91764.log 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72681.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.72681.log.gz --verbose 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr: -- /var/log/ceph/ceph-client.admin.80294.log 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91764.log.gz 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92254.log 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104913.log 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80294.log.gz 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70328.log 2026-02-21T22:25:52.670 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92254.log.gz 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108333.log 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104913.log.gz 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59544.log 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70328.log.gz 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr:gzip/var/log/ceph/ceph-client.admin.108333.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.109460.log 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108333.log.gz 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65671.log 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59544.log.gz 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79888.log 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109460.log.gz 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77634.log 2026-02-21T22:25:52.671 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65671.log.gz 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96273.log 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79888.log.gz 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59103.log 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77634.log.gz 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51394.log 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96273.log.gz 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107402.log 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59103.log.gz 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77095.log 2026-02-21T22:25:52.672 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51394.log.gz 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63661.log 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107402.log.gz 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112352.log 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77095.log.gz 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95979.log 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63661.log.gz 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65425.log 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112352.log.gz 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49660.log 2026-02-21T22:25:52.673 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95979.log.gz 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111372.log 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65425.log.gz 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79300.log 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49660.log.gz 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84757.log 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111372.log.gz 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59299.log 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79300.log.gz 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77536.log 2026-02-21T22:25:52.674 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84757.log.gz 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55709.log 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59299.log.gz 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62975.log 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77536.log.gz 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83139.log 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55709.log.gz 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56297.log 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62975.log.gz 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80637.log 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83139.log.gz 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90637.log 2026-02-21T22:25:52.675 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56297.log.gz 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90000.log 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80637.log.gz 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85982.log 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90637.log.gz 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80000.log 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90000.log.gz 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65572.log 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85982.log.gz 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80000.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.80000.log.gz -- 2026-02-21T22:25:52.676 INFO:teuthology.orchestra.run.trial060.stderr: /var/log/ceph/ceph-client.admin.74984.log 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78859.log 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65572.log.gz 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54631.log 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74984.log.gz 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95734.log 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78859.log.gz 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69543.log 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54631.log.gz 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61995.log 2026-02-21T22:25:52.677 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95734.log.gz 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77340.log 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69543.log.gz 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97990.log 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61995.log.gz 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82355.log 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77340.log.gz 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86472.log 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97990.log.gz 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85100.log 2026-02-21T22:25:52.678 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82355.log.gz 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80539.log 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86472.log.gz 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98186.log 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85100.log.gz 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66064.log 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80539.log.gz 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96371.log 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98186.log.gz 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111862.log 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66064.log.gz 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90588.log 2026-02-21T22:25:52.679 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96371.log.gz 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104547.log 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111862.log.gz 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68465.log 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90588.log.gz 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57473.log 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104547.log.gz 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83335.log 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68465.log.gz 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100944.log 2026-02-21T22:25:52.680 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57473.log.gz 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83335.log.gz 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91960.log 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58502.log 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100944.log.gz 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62240.log 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91960.log.gz 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78418.log 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58502.log.gz 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107206.log 2026-02-21T22:25:52.681 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62240.log.gz 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61113.log 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78418.log.gz 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61162.log 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107206.log.gz 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78271.log 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61113.log.gz 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65474.log 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61162.log.gz 2026-02-21T22:25:52.682 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107255.log 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78271.log.gz 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57130.log 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65474.log.gz 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95783.log 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107255.log.gz 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56787.log 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57130.log.gz 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56199.log 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95783.log.gz 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101728.log 2026-02-21T22:25:52.683 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56787.log.gz 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95097.log 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56199.log.gz 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71651.log 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101728.log.gz 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72092.log 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95097.log.gz 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101630.log 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71651.log.gz 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89853.log 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72092.log.gz 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2026-02-21T22:25:52.684 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101630.log.gz 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73808.log 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89853.log.gz 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82992.log 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85884.log 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73808.log.gz 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52719.log 2026-02-21T22:25:52.685 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82992.log.gz 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56248.log 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85884.log.gz 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97843.log 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52719.log.gz 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48591.log 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56248.log.gz 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52375.log 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97843.log.gz 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99706.log 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48591.log.gz 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98725.log 2026-02-21T22:25:52.686 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52375.log.gz 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99706.log.gz 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62730.log 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66603.log 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98725.log.gz 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70279.log 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62730.log.gz 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68367.log 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66603.log.gz 2026-02-21T22:25:52.687 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70279.log.gz 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72484.log 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111126.log 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68367.log.gz 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70475.log 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72484.log.gz 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74347.log 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111126.log.gz 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71504.log 2026-02-21T22:25:52.688 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70475.log.gz 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93284.log 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74347.log.gz 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79398.log 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71504.log.gz 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52571.log 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93284.log.gz 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58649.log 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79398.log.gz 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98284.log 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52571.log.gz 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75033.log 2026-02-21T22:25:52.689 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58649.log.gz 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76899.log 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98284.log.gz 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67730.log 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75033.log.gz 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61897.log 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76899.log.gz 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100441.log 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67730.log.gz 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51296.log 2026-02-21T22:25:52.690 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61897.log.gz 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66211.log 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100441.log.gz 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53013.log 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51296.log.gz 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48964.log 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66211.log.gz 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74886.log 2026-02-21T22:25:52.691 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53013.log.gz 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85247.log 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48964.log.gz 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75867.log 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74886.log.gz 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81421.log 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85247.log.gz 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101434.log 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75867.log.gz 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92941.log 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81421.log.gz 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71700.log 2026-02-21T22:25:52.692 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101434.log.gz 2026-02-21T22:25:52.693 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66652.log 2026-02-21T22:25:52.693 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92941.log.gz 2026-02-21T22:25:52.693 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63416.log 2026-02-21T22:25:52.693 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71700.log.gz 2026-02-21T22:25:52.693 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76507.log 2026-02-21T22:25:52.693 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66652.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96175.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63416.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56934.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76507.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88481.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96175.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94852.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56934.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101924.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88481.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97157.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94852.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51100.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101924.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95048.log 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97157.log.gz 2026-02-21T22:25:52.694 INFO:teuthology.orchestra.run.trial060.stderr:gzip/var/log/ceph/ceph-client.admin.51100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.110734.log 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51100.log.gz 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56150.log 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95048.log.gz 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57767.log 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110734.log.gz 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101581.log 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56150.log.gz 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78908.log 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57767.log.gz 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87501.log 2026-02-21T22:25:52.695 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101581.log.gz 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102316.log 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78908.log: /var/log/ceph/ceph-client.admin.87501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112646.log 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78908.log.gz 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87501.log.gz 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75082.log 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102316.log.gz 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81519.log 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112646.log.gz 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98529.log 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75082.log.gz 2026-02-21T22:25:52.696 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93725.log 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81519.log.gz 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67632.log 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98529.log.gz 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97402.log 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93725.log.gz 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101483.log 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67632.log.gz 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49277.log 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97402.log.gz 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101483.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T22:25:52.697 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101483.log.gz 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78712.log 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49277.log.gz 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62583.log 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94166.log 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78712.log.gz 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90882.log 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62583.log.gz 2026-02-21T22:25:52.698 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52179.log 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94166.log.gz 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64102.log 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90882.log.gz 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54533.log 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52179.log.gz 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75131.log 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64102.log.gz 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85835.log 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54533.log.gz 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83286.log 2026-02-21T22:25:52.699 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75131.log.gz 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90441.log 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85835.log.gz 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83286.log.gz 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90441.log.gz 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67534.log 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74102.log 2026-02-21T22:25:52.700 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55562.log 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67534.log.gz 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61211.log 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74102.log.gz 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108039.log 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55562.log.gz 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94362.log 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61211.log.gz 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr: 96.3%/var/log/ceph/ceph-client.admin.108039.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63024.log 2026-02-21T22:25:52.701 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108039.log.gz 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52522.log 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94362.log.gz 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79202.log 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63024.log.gz 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89755.log 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52522.log.gz 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108480.log 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79202.log.gz 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49081.log 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89755.log.gz 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111960.log 2026-02-21T22:25:52.702 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108480.log.gz 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88089.log 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49081.log.gz 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80245.log 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111960.log.gz 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56885.log 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88089.log.gz 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75524.log 2026-02-21T22:25:52.703 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80245.log.gz 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94656.log 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56885.log.gz 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110783.log 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75524.log.gz 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78467.log 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94656.log.gz 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110783.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110783.log.gz 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85639.log 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78467.log.gz 2026-02-21T22:25:52.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98235.log 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68269.log 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85639.log.gz 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69200.log 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98235.log.gz 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84953.log 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68269.log.gz 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69151.log 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69200.log.gz 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46695.log 2026-02-21T22:25:52.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84953.log.gz 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112303.log 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69151.log.gz 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.46695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46695.log.gz 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70034.log 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112303.log.gz 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97304.log 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95391.log 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70034.log.gz 2026-02-21T22:25:52.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101385.log 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97304.log.gz 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99755.log 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95391.log.gz 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106912.log 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101385.log.gz 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107843.log 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99755.log.gz 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.106912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108088.log 2026-02-21T22:25:52.707 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106912.log.gz 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52081.log 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107843.log.gz 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112107.log 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108088.log.gz 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84904.log 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52081.log.gz 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104472.log 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112107.log.gz 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2026-02-21T22:25:52.708 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84904.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68612.log 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104472.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69641.log 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83874.log 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68612.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91470.log 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69641.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84463.log 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83874.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91470.log.gz 2026-02-21T22:25:52.709 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44197.log 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109558.log 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84463.log.gz 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108529.log 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44197.log.gz 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64641.log 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109558.log.gz 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108529.log.gz 2026-02-21T22:25:52.710 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102463.log 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112205.log 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64641.log.gz 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107598.log 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102463.log.gz 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104008.log 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112205.log.gz 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93137.log 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107598.log.gz 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93235.log 2026-02-21T22:25:52.711 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104008.log.gz 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72730.log 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93137.log.gz 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63171.log 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93235.log.gz 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54974.log 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72730.log.gz 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60966.log 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63171.log.gz 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90833.log 2026-02-21T22:25:52.712 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54974.log.gz 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60966.log.gz 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96077.log 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90833.log.gz 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63269.log 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99461.log 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96077.log.gz/var/log/ceph/ceph-client.admin.63269.log: 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63269.log.gz 2026-02-21T22:25:52.713 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104548.log 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65770.log 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99461.log.gz 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93921.log 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104548.log.gz 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65278.log 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65770.log.gz 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64396.log 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93921.log.gz 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87011.log 2026-02-21T22:25:52.714 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65278.log.gz 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70132.log 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64396.log.gz 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102826.log 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87011.log.gz 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74690.log 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70132.log.gz 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58208.log 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102826.log.gz 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81127.log 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74690.log.gz 2026-02-21T22:25:52.715 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75818.log 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58208.log.gz 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81127.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81127.log.gz 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100294.log 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75818.log.gz 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56101.log 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87109.log 2026-02-21T22:25:52.716 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100294.log.gz 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103574.log 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56101.log.gz 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87109.log.gz 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72534.log 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103574.log.gz 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72534.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.72534.log.gz /var/log/ceph/ceph-client.admin.78565.log 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86423.log 2026-02-21T22:25:52.717 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87844.log 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78565.log.gz 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70867.log 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86423.log.gz 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85590.log 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87844.log.gz 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66995.log 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70867.log.gz 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70916.log 2026-02-21T22:25:52.718 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85590.log.gz 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64053.log 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66995.log.gz 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91862.log 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70916.log.gz 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99265.log 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64053.log.gz 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90196.log 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91862.log.gz 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.99265.log: -- /var/log/ceph/ceph-client.admin.65621.log 2026-02-21T22:25:52.719 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99265.log.gz 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49767.log 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90196.log.gz 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52866.log 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65621.log.gz 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49767.log.gz 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68661.log 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50904.log 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52866.log.gz 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66554.log 2026-02-21T22:25:52.720 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68661.log.gz 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80784.log 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50904.log.gz 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60426.log 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66554.log.gz 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77928.log 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80784.log.gz 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68906.log 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60426.log.gz 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97745.log 2026-02-21T22:25:52.721 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77928.log.gz 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111568.log 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68906.log.gz 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79251.log 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97745.log.gz 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82306.log 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111568.log.gz 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56395.log 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79251.log.gz 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82306.log.gz 2026-02-21T22:25:52.722 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109656.log 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76259.log 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56395.log.gz 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97549.log 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109656.log.gz 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52228.log 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76259.log.gz 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84561.log 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97549.log.gz 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52228.log.gz 2026-02-21T22:25:52.723 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82796.log 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74788.log 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84561.log.gz 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112450.log 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82796.log.gz 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103618.log 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74788.log.gz 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60377.log 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112450.log.gz 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103618.log.gz 2026-02-21T22:25:52.724 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71161.log 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95146.log 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60377.log.gz 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77977.log 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71161.log.gz 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68318.log 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95146.log.gz 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66946.log 2026-02-21T22:25:52.725 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77977.log.gz 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77438.log 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68318.log.gz 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57914.log 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66946.log.gz 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82845.log 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77438.log.gz 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84708.log 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57914.log.gz 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97598.log 2026-02-21T22:25:52.726 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82845.log.gz 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104276.log 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84708.log.gz 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97598.log.gz 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81963.log 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59152.log 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104276.log.gz 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66701.log 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81963.log.gz 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86129.log 2026-02-21T22:25:52.727 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59152.log.gz 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92450.log 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66701.log.gz 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107990.log 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86129.log.gz 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99314.log 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92450.log.gz 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90539.log 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107990.log.gz 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76458.log 2026-02-21T22:25:52.728 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99314.log.gz 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80833.log 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90539.log.gz 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75475.log 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76458.log.gz 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81372.log 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80833.log.gz 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75475.log.gz 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81816.log 2026-02-21T22:25:52.729 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81372.log.gz 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71308.log 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56983.log 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81816.log.gz 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108284.log 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71308.log.gz 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102512.log 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56983.log.gz 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93823.log 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108284.log.gz 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86521.log 2026-02-21T22:25:52.730 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102512.log.gz 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93676.log 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93823.log.gz 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101777.log 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86521.log.gz 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52620.log 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93676.log.gz 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53748.log 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101777.log.gz 2026-02-21T22:25:52.731 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107157.log 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52620.log.gz 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57816.log 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53748.log.gz 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87697.log 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107157.log.gz 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112499.log 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57816.log.gz 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71994.log 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87697.log.gz 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101336.log 2026-02-21T22:25:52.732 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112499.log.gz 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59985.log 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71994.log.gz 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56493.log 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101336.log.gz 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89510.log 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59985.log.gz 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91029.log 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56493.log.gz 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100686.log 2026-02-21T22:25:52.733 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89510.log.gz 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56689.log 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91029.log.gz 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59005.log 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100686.log.gz 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52032.log 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56689.log.gz 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73073.log 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59005.log.gz 2026-02-21T22:25:52.734 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49963.log 2026-02-21T22:25:52.735 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52032.log.gz 2026-02-21T22:25:52.735 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94117.log 2026-02-21T22:25:52.735 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73073.log.gz 2026-02-21T22:25:52.735 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93039.log 2026-02-21T22:25:52.735 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49963.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61260.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94117.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51345.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93039.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55170.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61260.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88138.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51345.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53993.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55170.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89902.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88138.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73220.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53993.log.gz 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84855.log 2026-02-21T22:25:52.736 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89902.log.gz 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91813.log 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73220.log.gz 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72632.log 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84855.log.gz 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61946.log 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91813.log.gz 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72632.log.gz 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101091.log 2026-02-21T22:25:52.737 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61946.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58257.log 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81865.log 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101091.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110685.log 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58257.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66260.log 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81865.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107696.log 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110685.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85443.log 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66260.log.gz 2026-02-21T22:25:52.738 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73318.log 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107696.log.gz 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110048.log 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85443.log.gz 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73318.log.gz 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76752.log 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58956.log 2026-02-21T22:25:52.739 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110048.log.gz 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62926.log 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76752.log.gz 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62191.log 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58956.log.gz 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69789.log 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62926.log.gz 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109362.log 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62191.log.gz 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98333.log 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69789.log.gz 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95587.log 2026-02-21T22:25:52.740 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109362.log.gz 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77585.log 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98333.log.gz 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99902.log 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95587.log.gz 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53895.log 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77585.log.gz 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92548.log 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99902.log.gz 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54140.log 2026-02-21T22:25:52.741 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53895.log.gz 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103222.log 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92548.log.gz 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48965.log 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54140.log.gz 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64298.log 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103222.log.gz 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49857.log 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48965.log.gz 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85345.log 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64298.log.gz 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49857.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.49857.log.gz --verbose 2026-02-21T22:25:52.742 INFO:teuthology.orchestra.run.trial060.stderr: -- /var/log/ceph/ceph-client.admin.69740.log 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73661.log 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85345.log.gz 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64543.log 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69740.log.gz 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91225.log 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73661.log.gz 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68563.log 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64543.log.gz 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84365.log 2026-02-21T22:25:52.743 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91225.log.gz 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53062.log 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68563.log.gz 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110636.log 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84365.log.gz 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53699.log 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53062.log.gz 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95293.log 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110636.log.gz 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61652.log 2026-02-21T22:25:52.744 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53699.log.gz 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67681.log 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95293.log.gz 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91617.log 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61652.log.gz 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57718.log 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67681.log.gz 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85737.log 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91617.log.gz 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62436.log 2026-02-21T22:25:52.745 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57718.log.gz 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95342.log 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85737.log.gz 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62436.log.gz 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104106.log 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95342.log.gz 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92646.log 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101875.log 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104106.log.gz 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2026-02-21T22:25:52.746 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92646.log.gz 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91274.log 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101875.log.gz 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100735.log 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76014.log 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91274.log.gz 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62338.log 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100735.log.gz 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76014.log.gz 2026-02-21T22:25:52.747 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54337.log 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52964.log 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62338.log.gz 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60230.log 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54337.log.gz 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88040.log 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52964.log.gz 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108137.log 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60230.log.gz 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88040.log.gz 2026-02-21T22:25:52.748 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100490.log 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61309.log 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108137.log.gz 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86962.log 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100490.log.gz 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66162.log 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61309.log.gz 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86962.log.gz 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57277.log 2026-02-21T22:25:52.749 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66162.log.gz 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102022.log 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95685.log 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57277.log.gz 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94754.log 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102022.log.gz 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66848.log 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95685.log.gz 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94754.log.gz 2026-02-21T22:25:52.750 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76406.log 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66848.log.gz 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72583.log 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76406.log.gz 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54484.log 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74151.log 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72583.log.gz 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67387.log 2026-02-21T22:25:52.751 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54484.log.gz 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88432.log 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74151.log.gz 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67387.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67387.log.gz -5 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50708.log 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66309.log 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88432.log.gz 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77242.log 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50708.log.gz 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66309.log.gz 2026-02-21T22:25:52.752 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88971.log 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71749.log 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77242.log.gz 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73024.log 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88971.log.gz 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58404.log 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71749.log.gz 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96912.log 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73024.log.gz 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69396.log 2026-02-21T22:25:52.753 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58404.log.gz 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81029.log 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96912.log.gz 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49453.log 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69396.log.gz 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82208.log 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81029.log.gz 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61554.log 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49453.log.gz 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109607.log 2026-02-21T22:25:52.754 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82208.log.gz 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94999.log 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61554.log.gz 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98676.log 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109607.log.gz 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94999.log.gz 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57669.log 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98676.log.gz 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98039.log 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91715.log 2026-02-21T22:25:52.755 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57669.log.gz 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58012.log 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98039.log.gz 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67926.log 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91715.log.gz 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62044.log 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58012.log.gz 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84266.log 2026-02-21T22:25:52.756 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67926.log.gz 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109852.log 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62044.log.gz 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94803.log 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84266.log.gz 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101532.log 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109852.log.gz 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109803.log 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94803.log.gz 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56738.log 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101532.log.gz 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85933.log 2026-02-21T22:25:52.757 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109803.log.gz 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73857.log 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56738.log.gz 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85933.log.gz 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55366.log 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73857.log.gz 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97696.log 2026-02-21T22:25:52.758 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55366.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38021.log 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55366.log.gz 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96224.log 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97696.log.gz 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph.tmp-client.admin.38021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59054.log 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38021.log.gz 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54680.log 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96224.log.gz 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57081.log 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59054.log.gz 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102120.log 2026-02-21T22:25:52.759 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54680.log.gz 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63563.log 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57081.log.gz 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48767.log 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102120.log.gz 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76654.log 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63563.log.gz 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61407.log 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48767.log.gz 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109509.log 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76654.log.gz 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61407.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T22:25:52.760 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61407.log.gz 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103310.log 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109509.log.gz 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108921.log 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78173.log 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103310.log.gz 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65917.log 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108921.log.gz 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98872.log 2026-02-21T22:25:52.761 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78173.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70181.log 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65917.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74739.log 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98872.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104962.log 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70181.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65721.log 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74739.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82747.log 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104962.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65721.log.gz 2026-02-21T22:25:52.762 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78026.log 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98088.log 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82747.log.gz 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90245.log 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78026.log.gz 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82061.log 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98088.log.gz 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107500.log 2026-02-21T22:25:52.763 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90245.log.gz 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.82061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94705.log 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82061.log.gz 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83531.log 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107500.log.gz 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70622.log 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94705.log.gz 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83384.log 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83531.log.gz 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60917.log 2026-02-21T22:25:52.764 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70622.log.gz 2026-02-21T22:25:52.765 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44458.log 2026-02-21T22:25:52.765 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83384.log.gz 2026-02-21T22:25:52.765 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75573.log 2026-02-21T22:25:52.765 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60917.log.gz 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79594.log 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44458.log.gz 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58551.log 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75573.log.gz 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49670.log 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79594.log.gz 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101679.log 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58551.log.gz 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97647.log 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49670.log.gz 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2026-02-21T22:25:52.766 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101679.log.gz 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75916.log 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97647.log.gz 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79692.log 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64249.log 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75916.log.gz 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86325.log 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79692.log.gz 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44705.log 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64249.log.gz 2026-02-21T22:25:52.767 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96716.log 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86325.log.gz 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.44705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58110.log 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44705.log.gz 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61358.log 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96716.log.gz 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71798.log 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58110.log.gz 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79104.log 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61358.log.gz 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92058.log 2026-02-21T22:25:52.768 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71798.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.79104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96961.log 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79104.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55121.log 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92058.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100993.log 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96961.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53111.log 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55121.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99118.log 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100993.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67142.log 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53111.log.gz 2026-02-21T22:25:52.769 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104325.log 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99118.log.gz 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83482.log 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67142.log.gz 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90686.log 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104325.log.gz 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49228.log 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83482.log.gz 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75377.log 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90686.log.gz 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2026-02-21T22:25:52.770 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49228.log.gz 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102708.log 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75377.log.gz 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.56003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53503.log 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102561.log 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102708.log.gz 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111224.log 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53503.log.gz 2026-02-21T22:25:52.771 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49179.log 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102561.log.gz 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100196.log 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111224.log.gz 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78810.log 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49179.log.gz 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94215.log 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100196.log.gz 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53797.log 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78810.log.gz 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66750.log 2026-02-21T22:25:52.772 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94215.log.gz 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67436.log 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53797.log.gz 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73514.log 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66750.log.gz 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104374.log 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67436.log.gz 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74298.log 2026-02-21T22:25:52.773 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73514.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77389.log 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104374.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.74298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85002.log 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74298.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99804.log 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77389.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93186.log 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85002.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88677.log 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99804.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93186.log.gz 2026-02-21T22:25:52.774 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107304.log 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67289.log 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88677.log.gz 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67975.log 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107304.log.gz 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63122.log 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67289.log.gz 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91323.log 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67975.log.gz 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63122.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63122.log.gz -5 2026-02-21T22:25:52.775 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59838.log 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67583.log 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91323.log.gz 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67191.log 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59838.log.gz 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86717.log 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67583.log.gz 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87452.log 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67191.log.gz 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86717.log.gz 2026-02-21T22:25:52.776 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76112.log 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92107.log 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87452.log.gz 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64445.log 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76112.log.gz 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60328.log 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92107.log.gz 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106961.log 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64445.log.gz 2026-02-21T22:25:52.777 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57620.log 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60328.log.gz 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.106961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91519.log 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106961.log.gz 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112401.log 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57620.log.gz 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59691.log 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91519.log.gz 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72239.log 2026-02-21T22:25:52.778 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112401.log.gz 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73955.log 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59691.log.gz 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78663.log 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72239.log.gz 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73955.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.73955.log.gz -5 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80588.log 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78222.log 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78663.log.gz 2026-02-21T22:25:52.779 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109166.log 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80588.log.gz 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81569.log 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78222.log.gz 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107353.log 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109166.log.gz 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110391.log 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81569.log.gz 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92499.log 2026-02-21T22:25:52.780 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107353.log.gz 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99363.log 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110391.log.gz 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48770.log 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92499.log.gz 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69004.log 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99363.log.gz 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65868.log 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48770.log.gz 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.69004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90980.log 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69004.log.gz 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.65868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100392.log 2026-02-21T22:25:52.781 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65868.log.gz 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77683.log 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90980.log.gz 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102914.log 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100392.log.gz 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77683.log: /var/log/ceph/ceph.log 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77683.log.gz 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48333.log 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102914.log.gz 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100147.log 2026-02-21T22:25:52.782 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95832.log 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48333.log.gz 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76801.log 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100147.log.gz 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71455.log 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95832.log.gz 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80441.log 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76801.log.gz 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109950.log 2026-02-21T22:25:52.783 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71455.log.gz 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88383.log 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80441.log.gz 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62142.log 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109950.log.gz 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81078.log 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88383.log.gz 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89314.log 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62142.log.gz 2026-02-21T22:25:52.784 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102169.log 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81078.log.gz 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100245.log 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89314.log.gz 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr: 96.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89363.log 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102169.log.gz 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93431.log 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100245.log.gz 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68073.log 2026-02-21T22:25:52.785 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89363.log.gz 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.93431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62877.log 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93431.log.gz 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81914.log 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68073.log.gz 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102267.log 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62877.log.gz 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54435.log 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81914.log.gz 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75965.log 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102267.log.gz 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62828.log 2026-02-21T22:25:52.786 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54435.log.gz 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.75965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62779.log 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75965.log.gz 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107059.log 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62828.log.gz 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57522.log 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62779.log.gz 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77291.log 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107059.log.gz 2026-02-21T22:25:52.787 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81176.log 2026-02-21T22:25:52.788 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57522.log.gz 2026-02-21T22:25:52.788 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49571.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77291.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60819.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81176.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62632.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49571.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57326.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60819.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67240.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62632.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107892.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57326.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77732.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67240.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104423.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107892.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.77732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66505.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77732.log.gz 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98137.log 2026-02-21T22:25:52.789 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104423.log.gz 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94411.log 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66505.log.gz 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59789.log 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98137.log.gz 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94313.log 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94411.log.gz 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.59789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101238.log 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59789.log.gz 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67338.log 2026-02-21T22:25:52.790 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94313.log.gz 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92695.log 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101238.log.gz 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57179.log 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67338.log.gz 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68122.log 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92695.log.gz 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.57179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66799.log 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57179.log.gz 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60770.log 2026-02-21T22:25:52.791 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68122.log.gz 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.66799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80049.log 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66799.log.gz 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76357.log 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60770.log.gz 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50806.log 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80049.log.gz 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97451.log 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76357.log.gz 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99167.log 2026-02-21T22:25:52.792 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50806.log.gz 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91666.log 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97451.log.gz 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99167.log.gz 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80392.log 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60181.log 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.91666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91666.log.gz 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108578.log 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80392.log.gz 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60181.log.gz 2026-02-21T22:25:52.793 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110097.log 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83972.log 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108578.log.gz 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102610.log 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110097.log.gz 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104743.log 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83972.log.gz 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.102610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101140.log 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102610.log.gz 2026-02-21T22:25:52.794 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73906.log 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104743.log.gz 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54042.log 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101140.log.gz 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86668.log 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73906.log.gz 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63612.log 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54042.log.gz 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112892.log 2026-02-21T22:25:52.795 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86668.log.gz 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110195.log 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63612.log.gz 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90735.log 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112892.log.gz 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103134.log 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110195.log.gz 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58907.log 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90735.log.gz 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103134.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103134.log.gz -5 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73122.log 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.58907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54827.log 0.0% 2026-02-21T22:25:52.796 INFO:teuthology.orchestra.run.trial060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58907.log.gz 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52915.log 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73122.log.gz 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.54827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89069.log 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54827.log.gz 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.52915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52915.log.gz 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86178.log 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.89069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64004.log 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89069.log.gz 2026-02-21T22:25:52.797 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.86178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90490.log 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86178.log.gz 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104841.log 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64004.log.gz 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.90490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112548.log 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90490.log.gz 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104841.log.gz 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112548.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112548.log.gz -5 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.88922.log 2026-02-21T22:25:52.798 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88285.log 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84316.log 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88922.log.gz 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.88285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87599.log 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88285.log.gz 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63710.log 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84316.log.gz 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87599.log.gz 2026-02-21T22:25:52.799 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96028.log 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.63710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98431.log 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63710.log.gz 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.96028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76063.log 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96028.log.gz 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107010.log 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98431.log.gz 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.76063.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.76063.log.gz 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.85394.log 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.107010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95636.log 2026-02-21T22:25:52.800 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107010.log.gz 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.85394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80490.log 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85394.log.gz 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81274.log 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95636.log.gz 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.80490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104178.log 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80490.log.gz 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104107.log 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81274.log.gz 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78320.log 2026-02-21T22:25:52.801 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104178.log.gz 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.104107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81620.log 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104107.log.gz 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.78320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72288.log 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78320.log.gz 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.81620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70377.log 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81620.log.gz 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97255.log 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72288.log.gz 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.70377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83580.log 2026-02-21T22:25:52.802 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70377.log.gz 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.97255.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.97255.log.gz /var/log/ceph/ceph-client.admin.50639.log 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.83580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67093.log 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83580.log.gz 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.50639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95244.log 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50639.log.gz 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.67093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100846.log 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67093.log.gz 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.95244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101042.log 2026-02-21T22:25:52.803 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95244.log.gz 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.100846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53650.log 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100846.log.gz 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.101042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64935.log 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101042.log.gz 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.53650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53650.log.gz 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.64935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61701.log 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64935.log.gz 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.48963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72926.log 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz 2026-02-21T22:25:52.804 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.61701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111617.log 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61701.log.gz 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103046.log 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72926.log.gz 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110293.log 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111617.log.gz 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111421.log 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103046.log.gz 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.110293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109754.log 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110293.log.gz 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49326.log 2026-02-21T22:25:52.805 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111421.log.gz 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.109754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99069.log 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109754.log.gz 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.49326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112058.log 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49326.log.gz 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108774.log 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99069.log.gz 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.112058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71357.log 2026-02-21T22:25:52.806 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112058.log.gz 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.108774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94460.log 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108774.log.gz 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.71357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60573.log 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71357.log.gz 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.94460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68024.log 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94460.log.gz 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99216.log 2026-02-21T22:25:52.807 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60573.log.gz 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98382.log 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.68024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68024.log.gz 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.99216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84806.log 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99216.log.gz 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.98382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73465.log 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98382.log.gz 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.84806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60524.log 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84806.log.gz 2026-02-21T22:25:52.808 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.73465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72828.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73465.log.gz 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.60524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55660.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60524.log.gz 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.72828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87060.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72828.log.gz 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.55660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111813.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55660.log.gz 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.87060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103737.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87060.log.gz 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.111813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92156.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111813.log.gz 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.103737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62485.log 2026-02-21T22:25:52.809 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103737.log.gz 2026-02-21T22:25:52.810 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.92156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92156.log.gz 2026-02-21T22:25:52.810 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.62485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62485.log.gz 2026-02-21T22:25:52.858 INFO:teuthology.orchestra.run.trial060.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T22:25:52.988 INFO:teuthology.orchestra.run.trial060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-21T22:25:53.172 INFO:teuthology.orchestra.run.trial060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-21T22:25:53.176 INFO:teuthology.orchestra.run.trial060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T22:26:06.491 INFO:teuthology.orchestra.run.trial060.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T22:26:07.651 INFO:teuthology.orchestra.run.trial060.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T22:26:09.947 INFO:teuthology.orchestra.run.trial060.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-21T22:26:11.386 INFO:teuthology.orchestra.run.trial060.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T22:26:11.387 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-21T22:26:11.388 INFO:teuthology.orchestra.run.trial060.stderr:real 0m18.870s 2026-02-21T22:26:11.388 INFO:teuthology.orchestra.run.trial060.stderr:user 1m4.048s 2026-02-21T22:26:11.388 INFO:teuthology.orchestra.run.trial060.stderr:sys 0m3.240s 2026-02-21T22:26:11.388 INFO:tasks.ceph:Archiving logs... 2026-02-21T22:26:11.388 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/log/ceph to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078/remote/trial060/log 2026-02-21T22:26:11.389 DEBUG:teuthology.orchestra.run.trial060:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T22:26:16.848 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T22:26:16.867 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T22:26:16.867 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T22:26:16.973 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T22:26:16.984 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T22:26:16.984 DEBUG:teuthology.orchestra.run.trial060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T22:26:17.000 INFO:teuthology.orchestra.run.trial060.stderr:bash: line 1: ntpq: command not found 2026-02-21T22:26:17.049 INFO:teuthology.orchestra.run.trial060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T22:26:17.049 INFO:teuthology.orchestra.run.trial060.stdout:=============================================================================== 2026-02-21T22:26:17.049 INFO:teuthology.orchestra.run.trial060.stdout:^- 23.186.168.133 2 6 377 13 +568us[ +568us] +/- 55ms 2026-02-21T22:26:17.049 INFO:teuthology.orchestra.run.trial060.stdout:^* pool-72-82-153-147.cmdnn> 1 7 377 77 -182us[ -281us] +/- 6893us 2026-02-21T22:26:17.049 INFO:teuthology.orchestra.run.trial060.stdout:^- 44.190.5.123 2 7 377 13 +1178us[+1178us] +/- 33ms 2026-02-21T22:26:17.049 INFO:teuthology.orchestra.run.trial060.stdout:^- 49.172.252.162.in-addr.a> 2 7 377 15 -1977us[-1977us] +/- 77ms 2026-02-21T22:26:17.050 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T22:26:17.061 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T22:26:17.061 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T22:26:17.073 DEBUG:teuthology.orchestra.run.trial060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T22:26:17.161 DEBUG:teuthology.orchestra.run.trial060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T22:26:17.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:26:17.188 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T22:26:17.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T22:26:17.212 INFO:teuthology.task.internal:Duration was 1525.474090 seconds 2026-02-21T22:26:17.212 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T22:26:17.227 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T22:26:17.227 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T22:26:17.265 INFO:teuthology.orchestra.run.trial060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T22:26:17.478 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T22:26:17.478 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial060.front.sepia.ceph.com 2026-02-21T22:26:17.478 DEBUG:teuthology.orchestra.run.trial060:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T22:26:17.535 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T22:26:17.535 DEBUG:teuthology.orchestra.run.trial060:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T22:26:17.653 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T22:26:17.653 DEBUG:teuthology.orchestra.run.trial060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T22:26:17.676 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:26:17.677 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:26:17.677 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T22:26:17.677 INFO:teuthology.orchestra.run.trial060.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T22:26:17.677 INFO:teuthology.orchestra.run.trial060.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T22:26:17.691 INFO:teuthology.orchestra.run.trial060.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T22:26:17.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T22:26:17.705 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T22:26:17.705 DEBUG:teuthology.orchestra.run.trial060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T22:26:17.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T22:26:17.769 DEBUG:teuthology.orchestra.run.trial060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T22:26:17.821 INFO:teuthology.orchestra.run.trial060.stdout:kernel.core_pattern = core 2026-02-21T22:26:17.840 DEBUG:teuthology.orchestra.run.trial060:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T22:26:17.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:26:17.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T22:26:17.913 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T22:26:17.914 DEBUG:teuthology.misc:Transferring archived files from trial060:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63078/remote/trial060 2026-02-21T22:26:17.914 DEBUG:teuthology.orchestra.run.trial060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T22:26:17.974 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T22:26:17.974 DEBUG:teuthology.orchestra.run.trial060:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T22:26:18.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T22:26:18.040 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T22:26:18.040 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T22:26:18.056 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T22:26:18.057 DEBUG:teuthology.orchestra.run.trial060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T22:26:18.079 INFO:teuthology.orchestra.run.trial060.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 22:26 /home/ubuntu/cephtest 2026-02-21T22:26:18.080 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T22:26:18.097 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T22:26:18.133 INFO:teuthology.run:Summary data: description: rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore/{alloc$/{stupid} base mem$/{normal-1} onode-segment$/{1M} write$/{v1/{compr$/{yes$/{zlib}} v1}}}} rados supported-random-distro$/{centos_latest}} duration: 1525.4740898609161 flavor: default owner: scheduled_nmordech@soko04.front.sepia.ceph.com success: true 2026-02-21T22:26:18.133 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T22:26:18.256 INFO:teuthology.run:pass